Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 7077a2c

Browse files
authoredMay 13, 2021
(AzureCXP) MicrosoftDocs/azure-docs#75234
Fixing all code snippets after "Define the input source" text are marked incorrectly with SQL instead of Scala.
1 parent 6d221d7 commit 7077a2c

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed
 

‎articles/hdinsight/spark/apache-spark-structured-streaming-overview.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ These JSON files are stored in the `temps` subfolder underneath the HDInsight c
6868

6969
First configure a DataFrame that describes the source of the data and any settings required by that source. This example draws from the JSON files in Azure Storage and applies a schema to them at read time.
7070

71-
```sql
71+
```scala
7272
import org.apache.spark.sql.types._
7373
import org.apache.spark.sql.functions._
7474

@@ -86,23 +86,23 @@ val streamingInputDF = spark.readStream.schema(jsonSchema).json(inputPath)
8686

8787
Next, apply a query that contains the desired operations against the Streaming DataFrame. In this case, an aggregation groups all the rows into 1-hour windows, and then computes the minimum, average, and maximum temperatures in that 1-hour window.
8888

89-
```sql
89+
```scala
9090
val streamingAggDF = streamingInputDF.groupBy(window($"time", "1 hour")).agg(min($"temp"), avg($"temp"), max($"temp"))
9191
```
9292

9393
### Define the output sink
9494

9595
Next, define the destination for the rows that are added to the results table within each trigger interval. This example just outputs all rows to an in-memory table `temps` that you can later query with SparkSQL. Complete output mode ensures that all rows for all windows are output every time.
9696

97-
```sql
97+
```scala
9898
val streamingOutDF = streamingAggDF.writeStream.format("memory").queryName("temps").outputMode("complete")
9999
```
100100

101101
### Start the query
102102

103103
Start the streaming query and run until a termination signal is received.
104104

105-
```sql
105+
```scala
106106
val query = streamingOutDF.start()
107107
```
108108

0 commit comments

Comments
 (0)
Please sign in to comment.