KAFKA-4730: Streams does not have an in-memory windowed store (#6239)
Implemented an in-memory window store allowing for range queries. A finite retention period defines how long records will be kept, ie the window of time for fetching, and the grace period defines the window within which late-arriving data may still be written to the store. Unit tests were written to test the functionality of the window store, including its insert/update/delete and fetch operations. Single-record, all records, and range fetch were tested, for both time ranges and key ranges. The logging and metrics for late-arriving (dropped)records were tested as well as the ability to restore from a changelog. Reviewers: John Roesler <john@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Showing
- docs/ops.html 1 addition, 0 deletionsdocs/ops.html
- docs/streams/developer-guide/processor-api.html 3 additions, 1 deletiondocs/streams/developer-guide/processor-api.html
- docs/streams/upgrade-guide.html 5 additions, 0 deletionsdocs/streams/upgrade-guide.html
- streams/src/main/java/org/apache/kafka/streams/state/Stores.java 39 additions, 1 deletion.../src/main/java/org/apache/kafka/streams/state/Stores.java
- streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryWindowBytesStoreSupplier.java 88 additions, 0 deletions...ams/state/internals/InMemoryWindowBytesStoreSupplier.java
- streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryWindowStore.java 393 additions, 0 deletions...he/kafka/streams/state/internals/InMemoryWindowStore.java
- streams/src/test/java/org/apache/kafka/streams/state/internals/InMemoryWindowStoreTest.java 476 additions, 0 deletions...afka/streams/state/internals/InMemoryWindowStoreTest.java
Loading
Please register or sign in to comment