Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
when using e.g.
asyncio.gather
to parallelize different operations in reality, the order in which firmware commands and responses appear in the validation json file depends on how fast the machine responds. this speed is different from the speed with which the validator-variants of the different io objects respond. for example, let's say backend 1 sends a request to machine 1 that takes 10 seconds to complete, and simultaneously backend 2 sends a request to backend 2 that only takes 5 seconds, the validation file might look like:but also:
even worse, when validating, we don't take into account the duration before a certain response was sent for a specific machine. doing so would mean validating a protocol takes as long as running it. that means: a certain validation backend might call
read
when the next line in the validation file is actually a different request or response. this means for the example above, backend 1 might issue aread
command which would get the next line which would beresponse2
.in the past, validation files were read linearly from start to finish, matching each request and response in the time they were saved.
to solve this, this PR introduces the option to read the requests and responses in order per device. that means that for every device (
io
object) we will still check that all requests and responses match and are in the correct order, but for different devices we can no longer synchronize them (when the option is enabled). this solves the case where asynchronous programming causes unpredictable validation files.sometimes it's important that commands to different machines are sent in a specific order. for example, opening the incubator door before moving a plate in. for this case, you can have the option disabled.
it is annoying because many protocols contain both cases: some parts require commands to different machines to be executed in a particular order, whereas other parts parallelize nicely. one example is after moving a plate: parking the iswap and closing the door can be parallelized.
one option to solve the general case is to place a lock on the return of a command to a machine until the machine of the item at the current index in the validation file (we would be running linearly) makes a request / reads a response. only after that do we release the lock. this would synchronize the code nicely i think. however, it wouldn't work for cases where multiple commands are sent in parallel to a singular machine (i am not sure if that is happening PLR right now, but it might in the future). i might implement that in a separate PR (perhaps merged after or instead of this one).