- Jan 12, 2019
-
-
Colin Patrick McCabe authored
This patch adds a framework to automatically generate the request/response classes for Kafka's protocol. The code will be updated to use the generated classes in follow-up patches. Below is a brief summary of the included components: **buildSrc/src** The message generator code is here. This code is automatically re-run by gradle when one of the schema files changes. The entire directory is processed at once to minimize the number of times we have to start a new JVM. We use Jackson to translate the JSON files into Java objects. **clients/src/main/java/org/apache/kafka/common/protocol/Message.java** This is the interface implemented by all automatically generated messages. **clients/src/main/java/org/apache/kafka/common/protocol/MessageUtil.java** Some utility functions used by the generated message code. **clients/src/main/java/org/apache/kafka/common/protocol/Readable.java, Writable.java, ByteBufferAccessor.java** The generated message code uses these classes for writing to a buffer. **clients/src/main/message/README.md** This README file explains how the JSON schemas work. **clients/src/main/message/\*.json** The JSON files in this directory implement every supported version of every Kafka API. The unit tests automatically validate that the generated schemas match the hand-written schemas in our code. Additionally, there are some things like request and response headers that have schemas here. **clients/src/main/java/org/apache/kafka/common/utils/ImplicitLinkedHashSet.java** I added an optimization here for empty sets. This is useful here because I want all messages to start with empty sets by default prior to being loaded with data. This is similar to the "empty list" optimizations in the `java.util.ArrayList` class. Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Ismael Juma <ismael@juma.me.uk>, Bob Barrett <bob.barrett@outlook.com>, Jason Gustafson <jason@confluent.io>
-
- Jun 12, 2018
-
-
Gitomain authored
KAFKA-6782: solved the bug of restoration of aborted messages for GlobalStateStore and KGlobalTable (#4900) Reviewer: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
-
- Apr 23, 2018
-
-
Debasish Ghosh authored
This PR implements a Scala wrapper library for Kafka Streams. The library is implemented as a project under streams, namely `:streams:streams-scala`. The PR contains the following: * the library implementation of the wrapper abstractions * the test suite * the changes in `build.gradle` to build the library jar The library has been tested running the tests as follows: ``` $ ./gradlew -Dtest.single=StreamToTableJoinScalaIntegrationTestImplicitSerdes streams:streams-scala:test $ ./gradlew -Dtest.single=StreamToTableJoinScalaIntegrationTestImplicitSerdesWithAvro streams:streams-scala:test $ ./gradlew -Dtest.single=WordCountTest streams:streams-scala:test ``` Author: Debasish Ghosh <ghosh.debasish@gmail.com> Author: Sean Glover <seglo@randonom.com> Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Ismael Juma <ismael@juma.me.uk>, John Roesler <john@confluent.io>, Damian Guy <damian@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #4756 from debasishg/scala-streams
-
- Mar 15, 2018
-
-
John Roesler authored
Some changes required to get the Streams system tests working via Docker To test: TC_PATHS="tests/kafkatest/tests/streams" bash tests/docker/run_tests.sh That command will take about 3.5 hours, and should pass. Note there are a couple of ignored tests. Reviewers: Guozhang Wang <wangguoz@gmail.com>, Bill Bejeck <bill@confluent.io>
-
- Aug 04, 2017
-
-
Abhishek Mendhekar authored
Author: Abhishek Mendhekar <amendhekar@linkedin.com> Reviewers: Joel Koshy <jjkoshy.w@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #3549 from abhishekmendhekar/KAFKA-5461
-
- Jul 20, 2017
-
-
Ismael Juma authored
I was investigating an exception in this code and found a few opportunities for making it clearer. I also added the `out` folder to `.gitignore` as IntelliJ sometimes uses that as the build folder. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com> Closes #3552 from ijuma/minor-quota-improvements
-
- May 10, 2017
-
-
Ewen Cheslack-Postava authored
Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Gwen Shapira Closes #2795 from ewencp/release-script and squashes the following commits: f1d0590 [Ewen Cheslack-Postava] Don't expose promotion to the user since it is not implemented yet. 1a6947a [Ewen Cheslack-Postava] Handle cleanup if there's a failure during generation of release notes. fa58401 [Ewen Cheslack-Postava] Fix hard-coded uses of trunk 639bcca [Ewen Cheslack-Postava] Try to cleanup after most failures. a3a7245 [Ewen Cheslack-Postava] Fix SCRIPT_DIR to be an absolute path so git clones against the REPO work when it is also your cwd de54c97 [Ewen Cheslack-Postava] Load/save preferences in a .release-settings.json file so you don't have to keep entering the same info repeatedly b559a61 [Ewen Cheslack-Postava] Check that the user doesn't have any oustanding diffs before starting the rest of the script ff0b330 [Ewen Cheslack-Postava] Store original starting branch to switch back to instead of using a default. b793562 [Ewen Cheslack-Postava] Use 2.12 instead of specific Scala version so we use the default 2.12 version. 382b7f9 [Ewen Cheslack-Postava] MINOR: Add a release script that helps generate release candidates.
-
- Feb 21, 2017
-
-
Christopher L. Shannon authored
build_eclipse is the configured output directory for eclipse when using the gradle eclipse plugin and should be ignored Author: Christopher L. Shannon <christopher.l.shannon@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2569 from cshannon/eclipse-gitignore
-
- Dec 20, 2016
-
-
Kamil Szymanski authored
Since jar files (including gradle-wrapper.jar) have to be excluded from source releases, current gradle wrapper infrastructure is incomplete rendering it unusable and confusing (I expected gradlew script to work without having to run default gradle task to download gradle-wrapper.jar). Author: Kamil Szymanski <kamil.szymanski.dev@gmail.com> Reviewers: Gwen Shapira <cshapi@gmail.com>, Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #1722 from kamilszymanski/gradlew-cleanup
-
- Aug 26, 2016
-
-
Shikhar Bhushan authored
Invoke the statusListener.onFailure() callback on start failures so that the statusBackingStore is updated. This involved a fix to the putSafe() functionality which prevented any update that was not preceded by a (non-safe) put() from completing, so here when a connector or task is transitioning directly to FAILED. Worker start methods can still throw if the same connector name or task ID is already registered with the worker, as this condition should not happen. Author: Shikhar Bhushan <shikhar@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1778 from shikhar/distherder-stayup-take4
-
- Aug 07, 2016
-
-
Kaufman Ng authored
added new consumer metrics section refactored common metrics into new section updated TOC Author: Kaufman Ng <kaufman@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1361 from coughman/KAFKA-3479-consumer-metrics-doc
-
- May 06, 2016
-
-
Geoff Anderson authored
This patch adds logic for the following: - remove hard-coded paths to various scripts and jars in kafkatest service classes - provide a mechanism for overriding path resolution logic with a "pluggable" path resolver class Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #1245 from granders/configurable-install-path
-
- Mar 10, 2016
-
-
Grant Henke authored
- Moves all generated docs under /docs/generated - Generates docs for Protocol, Errors, and ApiKeys - Adds new protocol.html page Author: Grant Henke <granthenke@gmail.com> Reviewers: Gwen Shapira Closes #970 from granthenke/protocol-doc-wip
-
- Nov 10, 2015
-
-
Ewen Cheslack-Postava authored
Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Gwen Shapira Closes #475 from ewencp/kafka-2379-connect-docs
-
- Nov 09, 2015
-
-
Grant Henke authored
Author: Grant Henke <granthenke@gmail.com> Reviewers: Guozhang Wang Closes #469 from granthenke/generated-configs
-
- Oct 02, 2015
-
-
Randall Hauch authored
The ` .metadata` and `.recommenders` keep IDE workspace state and should not be committed. Author: Randall Hauch <rhauch@gmail.com> Reviewers: Guozhang Wang Closes #259 from rhauch/kafka-2597
-
- Aug 03, 2015
-
-
Stevo Slavić authored
When running Kafka server from sources, logs directory gets created in root of repository, and kafka server logs end up there. Currently that directory is not ignored by git. This change adds root logs directory to .gitignore so that Kafka server logs are ignored and do not get tracked by git. Author: Stevo Slavić <sslavic@gmail.com> Reviewers: Ismael Juma Closes #94 from sslavic/patch-7 and squashes the following commits: c7b62a7 [Stevo Slavić] MINOR: Added to .gitignore Kafka server logs
-
- Jul 29, 2015
-
-
Geoff Anderson authored
Initial patch for KIP-25 Note that to install ducktape, do *not* use pip to install ducktape. Instead: ``` $ git clone gitgithub.com:confluentinc/ducktape.git $ cd ducktape $ python setup.py install ``` Author: Geoff Anderson <geoff@confluent.io> Author: Geoff <granders@gmail.com> Author: Liquan Pei <liquanpei@gmail.com> Reviewers: Ewen, Gwen, Jun, Guozhang Closes #70 from granders/KAFKA-2276 and squashes the following commits: a62fb6c [Geoff Anderson] fixed checkstyle errors a70f0f8 [Geoff Anderson] Merged in upstream trunk. 8b62019 [Geoff Anderson] Merged in upstream trunk. 47b7b64 [Geoff Anderson] Created separate tools jar so that the clients package does not pull in dependencies on the Jackson JSON tools or argparse4j. a9e6a14 [Geoff Anderson] Merged in upstream changes d18db7b [Geoff Anderson] fixed :rat errors (needed to add licenses) 321fdf8 [Geoff Anderson] Ignore tests/ and vagrant/ directories when running rat build task 795fc75 [Geoff Anderson] Merged in changes from upstream trunk. 1d93f06 [Geoff Anderson] Updated provisioning to use java 7 in light of KAFKA-2316 2ea4e29 [Geoff Anderson] Tweaked README, changed default log collection behavior on VerifiableProducer 0eb6fdc [Geoff Anderson] Merged in system-tests 69dd7be [Geoff Anderson] Merged in trunk 4034dd6 [Geoff Anderson] Merged in upstream trunk ede6450 [Geoff] Merge pull request #4 from confluentinc/move_muckrake 7751545 [Geoff Anderson] Corrected license headers e6d532f [Geoff Anderson] java 7 -> java 6 8c61e2d [Geoff Anderson] Reverted jdk back to 6 f14c507 [Geoff Anderson] Removed mode = "test" from Vagrantfile and Vagrantfile.local examples. Updated testing README to clarify aws setup. 98b7253 [Geoff Anderson] Updated consumer tests to pre-populate kafka logs e6a41f1 [Geoff Anderson] removed stray println b15b24f [Geoff Anderson] leftover KafkaBenchmark in super call 0f75187 [Geoff Anderson] Rmoved stray allow_fail. kafka_benchmark_test -> benchmark_test f469f84 [Geoff Anderson] Tweaked readme, added example Vagrantfile.local 3d73857 [Geoff Anderson] Merged downstream changes 42dcdb1 [Geoff Anderson] Tweaked behavior of stop_node, clean_node to generally fail fast 7f7c3e0 [Geoff Anderson] Updated setup.py for kafkatest c60125c [Geoff Anderson] TestEndToEndLatency -> EndToEndLatency 4f476fe [Geoff Anderson] Moved aws scripts to vagrant directory 5af88fc [Geoff Anderson] Updated README to include aws quickstart e5edf03 [Geoff Anderson] Updated example aws Vagrantfile.local 96533c3 [Geoff] Update aws-access-keys-commands 25a413d [Geoff] Update aws-example-Vagrantfile.local 884b20e [Geoff Anderson] Moved a bunch of files to kafkatest directory fc7c81c [Geoff Anderson] added setup.py 632be12 [Geoff] Merge pull request #3 from confluentinc/verbose-client 51a94fd [Geoff Anderson] Use argparse4j instead of joptsimple. ThroughputThrottler now has more intuitive behavior when targetThroughput is 0. a80a428 [Geoff Anderson] Added shell program for VerifiableProducer. d586fb0 [Geoff Anderson] Updated comments to reflect that throttler is not message-specific 6842ed1 [Geoff Anderson] left out a file from last commit 1228eef [Geoff Anderson] Renamed throttler 9100417 [Geoff Anderson] Updated command-line options for VerifiableProducer. Extracted throughput logic to make it reusable. 0a5de8e [Geoff Anderson] Fixed checkstyle errors. Changed name to VerifiableProducer. Added synchronization for thread safety on println statements. 475423b [Geoff Anderson] Convert class to string before adding to json object. bc009f2 [Geoff Anderson] Got rid of VerboseProducer in core (moved to clients) c0526fe [Geoff Anderson] Updates per review comments. 8b4b1f2 [Geoff Anderson] Minor updates to VerboseProducer 2777712 [Geoff Anderson] Added some metadata to producer output. da94b8c [Geoff Anderson] Added number of messages option. 07cd1c6 [Geoff Anderson] Added simple producer which prints status of produced messages to stdout. a278988 [Geoff Anderson] fixed typos f1914c3 [Liquan Pei] Merge pull request #2 from confluentinc/system_tests 81e4156 [Liquan Pei] Bootstrap Kafka system tests
-
- Mar 05, 2015
-
-
Joe Stein authored
-
- Feb 18, 2015
-
-
Tong Li authored
KAFKA-1960; .gitignore does not exclude test generated files and folders; reviewed by Joel Koshy and Gwen Shapira
-
- Dec 05, 2014
-
-
Joe Stein authored
KAFKA-1173 Using Vagrant to get up and running with Apache Kafka patch by Ewen Cheslack-Postava reviewed by Joe Stein
-
- Apr 09, 2014
-
-
Stevo Slavic authored
-
- Mar 19, 2014
-
-
Jay Kreps authored
-
- Dec 15, 2012
-
-
Derek Chen-Becker authored
Had to make some additional changes based on further mainline work. Leaving KafkaProject.scala in place as a reference for now.
-
- Oct 10, 2011
-
-
Chris Burroughs authored
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1180987 13f79535-47bb-0310-9956-ffa450edef68
-
- Oct 07, 2011
-
-
Neha Narkhede authored
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1180110 13f79535-47bb-0310-9956-ffa450edef68
-
- Aug 03, 2011
-
-
Chris Burroughs authored
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1153592 13f79535-47bb-0310-9956-ffa450edef68
-