modify the zero-size values in KJT for dynamic shape compatibility #2250
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
July 26, 2024 19:28
Summary: * Intention 1. The retention time for `unagi_newsie_delivery` and `unagi_row_newsie_delivery` tables are too short to monitor the traffic/volume abnormality. 2. Use ODS to log the following volume: `usecase`, `medium`, `policy`, `decision`, `reason` * Changes 1. add toODS for the following columns: `usecase`, `medium`, `policy`, `decision`, and `reason` 2. Fix a typo in prevous ods Differential Revision: D53558783
Summary: Pull Request resolved: pytorch#2202 # context * Discussed this ctx.new_dynamic_size() with Angela and she told me it's just to register a brand new dynamic shape for the batch_size. so I traced back why my registering batch_size as dynamic shape has so many problem. * all the complexity really comes from this line(s): serializer.py: batch_size = id_list_features.stride() * we get the batch_size from this function, which is derived from either "_maybe_compute_stride_kjt" or other sources. * Especially this line stride = lengths.numel() // len(keys), explained a lot of error messages I encountered before. * this correlation between the lengths.numel() and the batch_size really complicates the dynamic shape guards. for this I made up many artificial workarounds like: utils.py # details * we still declare/register the dynamic shape in the mark_dynamic_kjt function, * but pass the _dynamic_batch_size to kjt and retrieve it from the serializer's meta_forward. Differential Revision: D59201188
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D45410437 |
…ytorch#2250) Summary: Pull Request resolved: pytorch#2250 # context * dynamic shape usually has a minimum value requirement: `dynamic_shape >= 2` * however, in reality, the actual KJT._values could be empty * this issue was discribed in D57998381 > run the local-default every a few time one could get an error for some of the dynamic_shape being zero This is because in some corner case (not very rare though), the some dynamic_shape dim of the `sample_input` could be zero, and 0-size dynamic shape is handled differently during torch.export. **Bascially it will assume this dynamic shape should always be zero.** * error log: P1462233278 ``` [rank0]: - Not all values of vlen5 = L['args'][0][0].event_id_list_features_seqs['marketplace']._values.size()[0] in the specified range are valid because vlen5 was inferred to be a constant (0). [rank0]: Suggested fixes: [rank0]: vlen5 = 0 ``` # method * padding the kjt._values with the minimum required size `(2, )` * in the case of empty values, kjt._lengths and kjt._offsets should all be zeros * it doesn't affect the true logic/mathematic values of the kjt # issues 1. exported_program.module can't take in empty-value input. 2. deserialized unflattened model can't take in empty-value input, which could happen in real data. 3. deserialized unflattened model can't take in altered input, which could be a potential workaround if can't resolve pytorch#2. NOTE: Please check the in-line comments in the test file for details # Other Concerns 1. the inconsistency in the KJT (lengths are zeros, but values is non empty) might be incompatible with some downstream functions/operators, will need more tests to confirm. 2. Differential Revision: D45410437
|
This pull request was exported from Phabricator. Differential Revision: D45410437 |
TroyGarden
force-pushed
the
export-D45410437
branch
from
26dcce4 to
14f79a7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
context
dynamic_shape >= 2method
(2, )issues
NOTE: Please check the in-line comments in the test file for details
Other Concerns
Differential Revision: D45410437