Skip to content

Supporting multiple IIPs in one network? #77

@an0nym05

Description

@an0nym05

I'm trying to execute the following network:

'./sample.csv' -> FILE Source(csv/reader)
'./output.csv' -> FILE Target(csv/writer)
Source DATA -> DATA Target

But for some reason the network hangs and 2nd IIP never fired:

00:48:00 INF 1 run.go:29 > file="examples/simple.fbp" reading flow file
00:48:00 DBG 1 run.go:70 > IIP="'./sample.csv' -> FILE Source" adding IIP
00:48:00 DBG 1 run.go:70 > IIP="'./output.csv' -> FILE Target" adding IIP
00:48:00 DBG 10 csv.go:15 > CSVReader.Process()
00:48:00 DBG 12 csv.go:30 > CSVWriter.Process()
00:48:03 DBG 10 csv.go:18 > file="./sample.csv" CSVReader.Process()
00:48:03 DBG 12 csv.go:41 > file="" data="demo_table" CSVWriter.Process() 
00:48:03 DBG 10 csv.go:21 > CSVReader.Process() shutting down
...

And execution keeps waiting...

Expectation is to see the log line where both file and data variables are provided.

The components code is provided below:

package components

import (
	"time"

	"github.com/phuslu/log"
)

type CSVReader struct {
	File <-chan string
	Data chan<- string
}

func (reader *CSVReader) Process() {
	log.Debug().Msg("CSVReader.Process()")
	for file := range reader.File {
		time.Sleep(time.Second * 3)
		log.Debug().Str("file", file).Msg("CSVReader.Process()")
		reader.Data <- "demo_table"
	}
	log.Debug().Msg("CSVReader.Process() shutting down")
}

type CSVWriter struct {
	File <-chan string
	Data <-chan string
}

func (writer *CSVWriter) Process() {
	log.Debug().Msg("CSVWriter.Process()")
	defer func() {
		log.Debug().Msg("CSVWriter.Process() shutting down")
	}()
	var (
		file, table string
	)
	for {
		select {
		case file = <-writer.File:
			if table != "" {
				log.Debug().Str("file", file).Str("data", table).Msg("CSVWriter.Process() ")
				file = ""
				table = ""
			}
		case table = <-writer.Data:
			if file != "" {
				log.Debug().Str("file", file).Str("data", table).Msg("CSVWriter.Process() ")
				file = ""
				table = ""
			}
		}
	}

}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions