How to Generate Test Data in Confluent

In this section, we are going to be generate example record in Confluent platform , then run real time processing queries in order to analyse these records.

How to generate record ?

At first, Confluent platform should be started.

./confluent start

Step 1- Generate order data via Confluent data generator

./ksql-datagen quickstart=users

This is going to feed users_kafka_topic_json with sample data

Step 2- Register to the stream

In the KSQL Client, you need to register the stream to orders_topic

CREATE TABLE users_original (             \
    registertime BIGINT,                  \
    gender VARCHAR,                       \
    regionid VARCHAR,                     \
    userid VARCHAR)                       \
WITH (                                    \
    kafka_topic='users_kafka_topic_json', \
    value_format='JSON',                  \
    key = 'userid');

Step 3- Change the offset priority

This code should be run in KSQL

SET ‘auto.offset.reset’ = ‘earliest’;

Step 4- Run query

You can find two query snippets at the below

select * from users_original limit 10;
select gender,count(*) from users_original group By gender;

Final Words

KSQL is a real time data processing tool which going to help you analyse data in real time. In this section, we understand how KSQL accelerate your query