HPCC Report For Week 5

16 Sep HPCC Report For Week 5

This week has a great process on the project.

Finish Module

1. Test Data Preparation
   • Upload the test data and spray it to HPCC cluster
   • Convert each row to a JSON object and combine rows to JSON objects group by the trip
   • Save all the JSON objects of each trip to a new dataset
2. Test Harness
   • ECL program in HPCC reads the JSON messages from Kafka periodically

Processing Module

Build a simulated system which will transfer data from the Telematics system to HPCC system

1. Connect to Google resource. The firewall will be pushed on this Thursday night, but I check it failed at the morning on Friday and have sent feedback to the network team
2. Set up and configure the cluster
3. Create a topic for incoming telematics message
4. Push JSON messages from simulated Telematics system to Kafka
5. The ECL program at HPCC system will receive the JSON messages from Kafka, parse and save them as a dataset