Skip to content
Snippets Groups Projects
  • Colin Patrick McCabe's avatar
    a79d6dcd
    KAFKA-7793: Improve the Trogdor command line. (#6133) · a79d6dcd
    Colin Patrick McCabe authored
    
    * Allow the Trogdor agent to be started in "exec mode", where it simply
    runs a single task and exits after it is complete.
    
    * For AgentClient and CoordinatorClient, allow the user to pass the path
    to a file containing JSON, instead of specifying the JSON object in the
    command-line text itself.  This means that we can get rid of the bash
    scripts whose only function was to load task specs into a bash string
    and run a Trogdor command.
    
    * Print dates and times in a human-readable way, rather than as numbers
    of milliseconds.
    
    * When listing tasks or workers, output human-readable tables of
    information.
    
    * Allow the user to filter on task ID name, task ID pattern, or task
    state.
    
    * Support a --json flag to provide raw JSON output if desired.
    
    Reviewed-by: default avatarDavid Arthur &lt;mumrah@gmail.com&gt;, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>
    KAFKA-7793: Improve the Trogdor command line. (#6133)
    Colin Patrick McCabe authored
    
    * Allow the Trogdor agent to be started in "exec mode", where it simply
    runs a single task and exits after it is complete.
    
    * For AgentClient and CoordinatorClient, allow the user to pass the path
    to a file containing JSON, instead of specifying the JSON object in the
    command-line text itself.  This means that we can get rid of the bash
    scripts whose only function was to load task specs into a bash string
    and run a Trogdor command.
    
    * Print dates and times in a human-readable way, rather than as numbers
    of milliseconds.
    
    * When listing tasks or workers, output human-readable tables of
    information.
    
    * Allow the user to filter on task ID name, task ID pattern, or task
    state.
    
    * Support a --json flag to provide raw JSON output if desired.
    
    Reviewed-by: default avatarDavid Arthur &lt;mumrah@gmail.com&gt;, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>
TROGDOR.md 6.48 KiB

Trogdor

Trogdor is a test framework for Apache Kafka.

Trogdor can run benchmarks and other workloads. Trogdor can also inject faults in order to stress test the system.

Quickstart

First, we want to start a single-node Kafka cluster with a ZooKeeper and a broker.

Running ZooKeeper:

> ./bin/zookeeper-server-start.sh ./config/zookeeper.properties &> /tmp/zookeeper.log &

Running Kafka:

> ./bin/kafka-server-start.sh ./config/server.properties &> /tmp/kafka.log &

Then, we want to run a Trogdor Agent, plus a Trogdor broker.

To run the Trogdor Agent:

> ./bin/trogdor.sh agent -c ./config/trogdor.conf -n node0 &> /tmp/trogdor-agent.log &

To run the Trogdor Coordinator:

> ./bin/trogdor.sh coordinator -c ./config/trogdor.conf -n node0 &> /tmp/trogdor-coordinator.log &

Let's confirm that all of the daemons are running:

> jps
116212 Coordinator
115188 QuorumPeerMain
116571 Jps
115420 Kafka
115694 Agent

Now, we can submit a test job to Trogdor.

> ./bin/trogdor.sh client createTask -t localhost:8889 -i produce0 --spec ./tests/spec/simple_produce_bench.json
Sent CreateTaskRequest for task produce0.

We can run showTask to see what the task's status is:

> ./bin/trogdor.sh client showTask -t localhost:8889 -i produce0
Task bar of type org.apache.kafka.trogdor.workload.ProduceBenchSpec is DONE. FINISHED at 2019-01-09T20:38:22.039-08:00 after 6s