Support for specific source data formats
Challenge
Transmission system operators face a major challenge: numerous secondary systems, from digital relays, monitoring devices to IT platforms, generate enormous amounts of data.
This data comes in different and often incompatible formats, which makes its centralized analysis, optimization and management drastically difficult.
Solution
If the Cloudera platform does not support some of the source data formats, it is possible to upgrade the platform with internally developed applications for converting data from specific formats to formats supported by the Cloudera platform.
Examples of such solutions are:
An internally developed processor for Apache NiFi that converts files from COMTRADE format to JSON or Avro format suitable for subsequent transformations and analysis with tools available within the Cloudera platform.
An internally developed application designed to collect, decode TCP data from the final PDC (Phasor Data Concentrator) device that collects data from dozens of PMU (Phasor Measurement Unit) devices, and send the transformed data to a "kafka-topic", making it available for subsequent processing and analysis.
