Skip to content

Files

Latest commit

alexbuckgitalexbuckgit
[BULK UPDATE] DocuTune - Apply event-tier1-build-2022 ms.custom tag t…
May 23, 2022
55762f7 · May 23, 2022

History

History
163 lines (108 loc) · 8.45 KB

tutorial-java-spring-confluent-kafka.md

File metadata and controls

163 lines (108 loc) · 8.45 KB
title description ms.devlang ms.custom author ms.author ms.service ms.topic ms.date
Tutorial: Deploy a Spring Boot app connected to Apache Kafka on Confluent Cloud with Service Connector in Azure Spring Cloud
Create a Spring Boot app connected to Apache Kafka on Confluent Cloud with Service Connector in Azure Spring Cloud.
java
event-tier1-build-2022
shizn
xshi
service-connector
tutorial
05/03/2022

Tutorial: Deploy a Spring Boot app connected to Apache Kafka on Confluent Cloud with Service Connector in Azure Spring Cloud

Learn how to access Apache Kafka on Confluent Cloud for a Spring Boot application running on Azure Spring Cloud. In this tutorial, you complete the following tasks:

[!div class="checklist"]

  • Create Apache Kafka on Confluent Cloud
  • Create a Spring Cloud application
  • Build and deploy the Spring Boot app
  • Connect Apache Kafka on Confluent Cloud to Azure Spring Cloud using Service Connector

Set up your initial environment

  1. Have an Azure account with an active subscription. Create an account for free.
  2. Install Java 8 or 11.
  3. Install the Azure CLI 2.18.0 or higher, with which you run commands in any shell to provision and configure Azure resources.

Clone or download the sample app

  1. Clone the sample repository:

    git clone https://github.com/Azure-Samples/serviceconnector-springcloud-confluent-springboot/

  2. Navigate into that folder:

    cd serviceconnector-springcloud-confluent-springboot

Prepare cloud services

Create an instance of Apache Kafka for Confluent Cloud

Create an instance of Apache Kafka for Confluent Cloud by following this guidance.

Create Kafka cluster and schema registry on Confluent Cloud

  1. Sign in to Confluent Cloud using the SSO provided by Azure

    :::image type="content" source="media/tutorial-java-spring-confluent-kafka/azure-confluent-sso-login.png" alt-text="The link of Confluent cloud SSO login using Azure portal" lightbox="media/tutorial-java-spring-confluent-kafka/azure-confluent-sso-login.png":::

  2. Use the default environment or create a new one

    :::image type="content" source="media/tutorial-java-spring-confluent-kafka/confluent-cloud-env.png" alt-text="Cloud environment of Apache Kafka on Confluent Cloud" lightbox="media/tutorial-java-spring-confluent-kafka/confluent-cloud-env.png":::

  3. Create a Kafka cluster with the following information

    • Cluster type: Standard
    • Region/zones: eastus(Virginia), Single Zone
    • Cluster name: cluster_1 or any other name.

  4. In Cluster overview -> Cluster settings, get the Kafka bootstrap server url and take note it down.

    :::image type="content" source="media/tutorial-java-spring-confluent-kafka/confluent-cluster-setting.png" alt-text="Cluster settings of Apache Kafka on Confluent Cloud" lightbox="media/tutorial-java-spring-confluent-kafka/confluent-cluster-setting.png":::

  5. Create API keys for the cluster in Data integration -> API Keys -> + Add Key with Global access. Note down the key and secret.

  6. Create a topic named test with partitions 6 in Topics -> + Add topic

  7. Under default environment, select the Schema Registry tab. Enable the Schema Registry and note down the API endpoint.

  8. Create API keys for schema registry. Save the key and secret.

Create a Spring Cloud instance

Create an instance of Azure Spring Cloud by following the Spring Cloud quickstart in Java. Make sure your Spring Cloud instance is created in the region that has Service Connector support.

Build and deploy the app

Build the sample app and create a new spring app

  1. Sign in to Azure and choose your subscription.

    az login

az account set --subscription <Name or ID of your subscription>

  2. Build the project using gradle

    ./gradlew build

  3. Create the app with a public endpoint assigned. If you selected Java version 11 when generating the Spring Cloud project, include the --runtime-version=Java_11 switch.

    az spring-cloud app create -n hellospring -s <service-instance-name> -g <your-resource-group-name> --assign-endpoint true

Create service connection using Service Connector

CLI

Run the following command to connect your Apache Kafka on Confluent Cloud to your spring cloud app.

az spring-cloud connection create confluent-cloud -g <your-spring-cloud-resource-group> --service <your-spring-cloud-service> --app <your-spring-cloud-app> --deployment <your-spring-cloud-deployment> --bootstrap-server <kafka-bootstrap-server-url> --kafka-key <cluster-api-key> --kafka-secret <cluster-api-secret> --schema-registry <kafka-schema-registry-endpoint> --schema-key <registry-api-key> --schema-secret <registry-api-secret>

Replace the following placeholder texts with your own data:

  • Replace <your-resource-group-name> with the resource group name that you created your Spring Cloud instance.
  • Replace <kafka-bootstrap-server-url> with your kafka bootstrap server url (the value should be like pkc-xxxx.eastus.azure.confluent.cloud:9092)
  • Replace <cluster-api-key> and <cluster-api-secret> with your cluster API key and secret.
  • Replace <kafka-schema-registry-endpoint> with your kafka Schema Registry endpoint (the value should be like https://psrc-xxxx.westus2.azure.confluent.cloud)
  • Replace <registry-api-key> and <registry-api-secret> with your kafka Schema Registry API key and secret.

Note

If you see the error message "The subscription is not registered to use Microsoft.ServiceLinker", please run az provider register -n Microsoft.ServiceLinker to register the Service Connector resource provider and run the connection command again.

Portal

Select Service Connector and enter the following settings.

Setting Suggested value Description
Service Type Apache Kafka on Confluent cloud Target service type. If you don't have an Apache Kafka on Confluent Cloud target service, complete the previous steps in this tutorial.
Name Generated unique name The connection name that identifies the connection between your Spring Cloud and target service.
Kafka bootstrap server url Your Kafka bootstrap server url. Enter the value from earlier step: "Create Kafka cluster and schema registry on Confluent Cloud".
Cluster API Key Your cluster API key. Your cluster API key.
Cluster API Secret Your cluster API secret. Your cluster API secret.
Create connection for schema registry Checked Also create a connection to the schema registry.
Schema Registry endpoint Your Kafka Schema Registry endpoint.
Schema Registry API Key Your Kafka Schema Registry API Key. Your Kafka Schema Registry API Key.
Schema Registry API Secret Your Kafka Schema Registry API Secret. Your Kafka Schema Registry API Secret.

Select Review + Create to review the connection settings. Then select Create to create start creating the service connection.

Deploy the JAR file

Run the following command to upload the JAR file (build/libs/java-springboot-0.0.1-SNAPSHOT.jar) to your Spring Cloud app.

az spring-cloud app deploy -n hellospring -s <service-instance-name> -g <your-resource-group-name>  --artifact-path build/libs/java-springboot-0.0.1-SNAPSHOT.jar

Validate the Kafka data ingestion

Navigate to your Spring Cloud app's endpoint from the Azure portal and select the application URL. You'll see "10 messages were produced to topic test".

Then go to the Confluent portal and the topic's page will show production throughput.

:::image type="content" source="media/tutorial-java-spring-confluent-kafka/confluent-sample-metrics.png" alt-text="Sample metrics" lightbox="media/tutorial-java-spring-confluent-kafka/confluent-sample-metrics.png":::

Next steps

Follow the tutorials listed below to learn more about Service Connector.

[!div class="nextstepaction"] Learn about Service Connector concepts