Template actions invoked more than one time in a request

[alexgarzao]

Is your feature request related to a problem? Please describe.

I have a use case where a request is composed of a serie with N events. For example:

{
    "SerieID":"94083c9c-0683-4407-a00c-58b9787aca83",
    "Time":1597388282, 
    "Events":[
        {
            "EventID":"8881c0cc-6d62-4d2a-9a81-36dbfd1e2f1c",
            "EventType":"credit",
            "ObjectKey":"f95f8288-8f1f-4338-bb68-53a1cd61c25f"
        },
        {
            "EventID":"6491b357-8163-4cc7-bc8c-af3b3c6b92f9",
            "EventType":"debit",
            "ObjectKey":"75221e12-bccb-4010-ac3e-41b0aa5f9dad"
        }
    ]
}

EventID must be unique, in a request.

I tried to pass the following parameter to ghz cli:

$ ghz ... -d '{"SerieID":"{{.UUID}}", "Time":"{{.Timestamp}}", "Events":[{"EventID":"{{.UUID}}", "EventType":"credit", "ObjectKey":"f95f8288-8f1f-4338-bb68-53a1cd61c25f"},{"EventID":"{{.UUID}}", "EventType":"debit", "ObjectKey":"75221e12-bccb-4010-ac3e-41b0aa5f9dad"}]}'

In my use case, each EventID must be unique. Reading the docs, I understood that UUID template action is unique for each call invocation/request.

Describe the solution you'd like
In a request, for each "{{.UUID}}", a new UUID is generated.

Describe alternatives you've considered
I tried to implement using the package, with "runner.WithDataFromReader". But WithDataFromReader reads all the stream at the start.

With few series (10K), it's ok, but with 500K, isn't feasible to load all the stream in memory.

[Streppel]

@bojand could you take a look at this? maybe we are doing something wrong here and haven't noticed it yet

[bojand]

Hello, it would have been helpful to post the proto definition, as I am not sure if I am understanding the issue and question correctly. But if I am understanding it right, the entire request message is like the JSON posted above. Given that, every call will get new template data and be filled with it. That is every request gets new data, new UUID in your case. We do not generate new template values for each occurrence of the template variable within the template. Hope this makes the distinction clear.
So provided data like:

{
  "SerieID": "{{.UUID}}",
  "Time": "{{.Timestamp}}",
  "Events": [
    {
      "EventID": "{{.UUID}}",
      "EventType": "credit",
      "ObjectKey": "f95f8288-8f1f-4338-bb68-53a1cd61c25f"
    },
    {
      "EventID": "{{.UUID}}",
      "EventType": "debit",
      "ObjectKey": "75221e12-bccb-4010-ac3e-41b0aa5f9dad"
    }
  ]
}

First call might get data like:

{
  "SerieID": "6d3a9ab1-7310-4964-8338-fdeb8c4b960f",
  "Time": "{{.Timestamp}}",
  "Events": [
    {
      "EventID": "6d3a9ab1-7310-4964-8338-fdeb8c4b960f",
      "EventType": "credit",
      "ObjectKey": "f95f8288-8f1f-4338-bb68-53a1cd61c25f"
    },
    {
      "EventID": "6d3a9ab1-7310-4964-8338-fdeb8c4b960f",
      "EventType": "debit",
      "ObjectKey": "75221e12-bccb-4010-ac3e-41b0aa5f9dad"
    }
  ]
}

2nd call we create will get new UUID:

{
  "SerieID": "3ded7836-b860-43cd-b88b-3ae78d67c834",
  "Time": "{{.Timestamp}}",
  "Events": [
    {
      "EventID": "3ded7836-b860-43cd-b88b-3ae78d67c834",
      "EventType": "credit",
      "ObjectKey": "f95f8288-8f1f-4338-bb68-53a1cd61c25f"
    },
    {
      "EventID": "3ded7836-b860-43cd-b88b-3ae78d67c834",
      "EventType": "debit",
      "ObjectKey": "75221e12-bccb-4010-ac3e-41b0aa5f9dad"
    }
  ]
}

and so on... The timestamp would be filled in and I am ignoring it for purpose of demonstration.
Note that request data for each call will get new template variable values. But just a single {{.UUID}} within the call will be generated.

If you desire a new uuid within a single call that's not something ghz can accomplish now. I think the simplest way we could support that would be to add a template function, something like newUUID. Where then the data would look like:

{
  "SerieID": "{{newUUID}}",
  "Time": "{{.Timestamp}}",
  "Events": [
    {
      "EventID": "{{newUUID}}",
      "EventType": "credit",
      "ObjectKey": "f95f8288-8f1f-4338-bb68-53a1cd61c25f"
    },
    {
      "EventID": "{{newUUID}}",
      "EventType": "debit",
      "ObjectKey": "75221e12-bccb-4010-ac3e-41b0aa5f9dad"
    }
  ]
}

And the newUUID function would be called each time to generate a new UUID.
Is this more in line to the behaviour you're looking for?

[alexgarzao]

Hello @bojand !

Before all, thanks for your attention :-)

Yep, this approach, invoking a template function like 'newUUID', is perfect for my use case.

I looked at the ghz source, and the template.New function is called inside 'callTemplateData.execute(data string)'.

Is there a way to pass a template with 'custom template functions' to ghz, in runner.Run? Something like that:

// my custom template
funcMap := template.FuncMap{
	"newUUID": func() string {
		return uuid.New.String()
	},
}

tmpl := template.New("titleTest").Funcs(funcMap)
if err != nil {
	log.Fatalf("parsing: %s", err)
}

report, err := runner.Run(
	runner.WithTemplate(tmpl),   // HERE MY  CUSTOM TEMPLATE...
	"proto.Storm.SaveSerie",
	"0.0.0.0:9000",
	runner.WithProtoFile("storm.proto", []string{}),
	runner.WithDataFromReader(scenario),
	runner.WithInsecure(true),
	runner.WithTotalRequests(totalRequests),
	runner.WithConcurrency(concurrency),
)

[bojand]

Hello this should be resolved in 0.58.0. Please see the changelog for details.

[alexgarzao]

I tried to use now, and worked very well :-)

Thanks a lot for your help.