Skip to content

Conversation

@sananguliyev
Copy link
Contributor

@sananguliyev sananguliyev commented May 27, 2025

In the current build of the traced stream, tracing events are currently grouped only by section. This setup makes it difficult to link these events together and understand the complete message journey in the stream flow. The new implementation addresses this issue by:

  • Assigning a unique flow ID to each message journey
  • Adding a timestamp to each tracing event to provide a clearer order of the steps the message went through
  • Introducing EventsByFlowID, which organizes events by flow and arranges them in chronological order based on timestamp.

Additionally: If no tags exist: git describe --tags will fail, but the error message will be suppressed by 2>/dev/null, and then it will fall back to "v0.0.0". Currently, it prints fatal: No names found, cannot describe anything. when make file tries to generate version.

Fixes #344

@sananguliyev sananguliyev changed the title Set unique id & timestamp to each flow Set unique flow id & timestamp to each flow May 27, 2025
@sananguliyev
Copy link
Contributor Author

@jem-davies @gregfurman
Could you please take a look when you have time?

@gregfurman
Copy link
Collaborator

@sananguliyev Hey! Thanks for the contribution and apologies for the delay. Will take a look this eve 😄

Copy link
Collaborator

@gregfurman gregfurman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions relating to the metadata approach. Think better internal tracing is a brilliant idea so I'm keen to hear your thoughts on my comments 😄

@gregfurman gregfurman marked this pull request as draft October 18, 2025 17:13
@sananguliyev sananguliyev marked this pull request as ready for review October 18, 2025 22:08
Copy link
Collaborator

@gregfurman gregfurman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow ups on this!

I'm thinking whether we want to ever set the flowID values outside of the traceInput (i.e when a message part is first created), although I suppose there's the case where we want to create child-spans.

if t.e.IsEnabled() {
_ = tran.Payload.Iter(func(i int, part *message.Part) error {
_ = atomic.AddUint64(t.ctr, 1)
t.e.Add(EventProduceOf(part))
return nil
})

Let me know!

@sananguliyev
Copy link
Contributor Author

sananguliyev commented Dec 11, 2025

Thanks for the follow ups on this!

I'm thinking whether we want to ever set the flowID values outside of the traceInput (i.e when a message part is first created), although I suppose there's the case where we want to create child-spans.

if t.e.IsEnabled() {
_ = tran.Payload.Iter(func(i int, part *message.Part) error {
_ = atomic.AddUint64(t.ctr, 1)
t.e.Add(EventProduceOf(part))
return nil
})

Let me know!

Why We Need Both

  1. flowIDInput: Ensures flow IDs are always available (even without tracing)
  2. tracedInput: Records what happens with those flow IDs (only when user wants tracing)

  • flowIDInput = "Give every message a passport (flow ID)"
  • tracedInput = "Write down in a log book every time a message passes through"

And this also helps to group events not only by section but also by flow.

Please, take another look. Thanks in advance. @gregfurman

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Assign unique flow ID to each message

2 participants