Skip to content

Multiple output streams for trident - #731

Open
alienrobotwizard wants to merge 2 commits into
nathanmarz:masterfrom
alienrobotwizard:trident-mos
Open

Multiple output streams for trident#731
alienrobotwizard wants to merge 2 commits into
nathanmarz:masterfrom
alienrobotwizard:trident-mos

Conversation

@alienrobotwizard

Copy link
Copy Markdown

Addresses issue 638 (http://github.com/nathanmarz/storm/issues/638). @mridulj @mrflip

Consider:

Map<String, Fields> outputStreamMapping = new HashMap<String, Fields>();
outputStreamMapping.put("streamA", new Fields("field_in_a"));
outputStreamMapping.put("streamB", new Fields("field_in_b"));

Stream s = topology.newStream("testStream", spout);
StreamCollection multiStream = s.each(
  s.getOutputFields(),
  new DebugMultiStream("streamA", "streamB"),
  new StreamMap(outputStreamMapping)            
  );

Stream streamA = multiStream.getStream("streamA")
streamA.each(new Fields("field_in_a"), new Debug());

@nathanmarz

Copy link
Copy Markdown
Owner

I've only glanced at this, but I have two initial comments:

  1. This needs tests
  2. I'm not so sure about the API. I think that when the multistream each is used, it should return a "MultiStream" object instead of a stream object. And then the MultiStream object can be queried for the particular streams.

@alienrobotwizard

Copy link
Copy Markdown
Author

Nathan,

  1. Yes, absolutely. That's the next thing. I just wanted to get the patch out asap since @mridulj was asking about it.
  2. The multi each returns a StreamCollection object that has a 'getStream' method. It could just as easily be called MultiStream if that seems like a clearer name.

@alienrobotwizard

Copy link
Copy Markdown
Author

Oh I see the confusion. I've been writing my topologies in jruby which is why I can reuse 's' up there for both a Stream and a StreamCollection. I just did a quick conversion as I was typing. But my previous comment holds, it does return a collection in the way you're asking,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants