Skip to content

Copy iterator interrupt - #681

Open
abudnik wants to merge 7 commits into
reverbrain:masterfrom
abudnik:copy_iter_interrupt
Open

Copy iterator interrupt#681
abudnik wants to merge 7 commits into
reverbrain:masterfrom
abudnik:copy_iter_interrupt

Conversation

@abudnik

@abudnik abudnik commented Dec 22, 2015

Copy link
Copy Markdown
Member
  1. copy-iterator: stop iteration only if connection failed or no space left on remote backend
  2. server-send: check that source and destination backends are not the same
  3. server-send: added 80 reserved bytes to dnet_server_send_request struct

@abudnik

abudnik commented Dec 22, 2015

Copy link
Copy Markdown
Member Author

PRs #668 & #679 are depend on this PR

Comment thread library/dnet.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be on the client side?
Do not even try to start iterator if its source/destination backends are the same?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this happens when client state gone during iteration. It starts to write to itself.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see. Please add extended comments about this case

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@bioothod

Copy link
Copy Markdown
Member

How do #668 and #679 depend on this pull request?

Comment thread library/dnet.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it really interrupt iterator in this case or it just skips the key?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It interrupts iterrator. I've manually tested this code with iptables before sending PR :)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can it be tested via iptables? It's about writing keys when destination and source are the same backend.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It interrupts iterrator.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iterator's callback returns error code from dnet_server_send_write(), so iteration stops on this error.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested manually with iptables closing remote connection, so this case was reproduced and iteration stopped.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't interrupt iteration here and there are few reasons for that:

  • it should avoid writing keys to the same backend, but it should write all keys to other groups
  • if there is only one destination group, it should skip keys that should be written to the same backend, but it should write all keys to other backends

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and will fix it soon.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abudnik
abudnik force-pushed the copy_iter_interrupt branch from f5af2a3 to ae161ba Compare December 23, 2015 15:17
Comment thread library/dnet.c Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires more comments about the whole logic behind this function.
Why is it needed, what does it do, how does it modify group array.

One should understand this function without looking into the code, but from description.
Looking into the code is only needed when you want to fix a bug.

It also leaks state.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abudnik
abudnik force-pushed the copy_iter_interrupt branch 2 times, most recently from 4782b72 to 2b37492 Compare December 24, 2015 13:07
@abudnik
abudnik force-pushed the copy_iter_interrupt branch from 2b37492 to 62b1d29 Compare January 11, 2016 13:51
@abudnik abudnik mentioned this pull request Jan 13, 2016
@abudnik
abudnik force-pushed the copy_iter_interrupt branch from c78ce00 to c21d415 Compare September 26, 2016 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants