All of the components that use kafka have been tested with kafka_2.11-0.10.0.1 and zookeeper-3.4.8 both under java 8 on Amazon Linux

Provision Zookeeper instances using default configuration

Provision Kafka instances using default configuration but with the following changes

1. change the broker.id

2. listeners = PLAINTEXT://:9092

3. host.name=

4. advertised.host.name=<ip

5. num.partitions=2 (or = to number of logsession/logshuttle instances)

6. log.retention.hours=1 (remove any other log retention values)

7. zookeeper.connect=:2181,:2181,:2181/kafka (last /kafka is really important as it will be required in the fluentd config)