Page History

...

cat /var/log/squid/access.log | ${HDP_HOME}/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic squid

${HDP_HOME}/kafka-broker/bin/kafka-console-consumer.sh --zookeeper $ZOOKEEPER bootstrap-server $BROKERLIST --topic squid --from-beginning

...

1. Sets up default mappings for metron-specific types, e.g. timestamps.
2. Sets up types for properties that will come from the parsed data, e.g. ip_src_addr.

If you're using the Full dev environment, you might want to stop some of the other parsers to free up resources.

for parser in bro__snort__yaf profiler pcap batch_indexing; do storm kill parser; done

Now start the new squid parser topology:

...