Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit dbac6f8

Browse files
authoredApr 22, 2022
Update apache-spark-troubleshoot-event-log-requestbodytoolarge.md
Just remove docs.microsoft.com and start with /rest
1 parent adc27dc commit dbac6f8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed
 

‎articles/hdinsight/spark/apache-spark-troubleshoot-event-log-requestbodytoolarge.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ java.io.IOException: Operation failed: "The request body is too large and exceed
3535

3636
## Cause
3737

38-
Files created over ABFS driver create Block blobs in Azure storage. Your Spark event log file is probably hitting the file length limit for WASB. See [50,000 blocks that a block blob can hold at max](https://docs.microsoft.com/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs).
38+
Files created over ABFS driver create Block blobs in Azure storage. Your Spark event log file is probably hitting the file length limit for WASB. See [50,000 blocks that a block blob can hold at max](/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs).
3939

4040
In Spark 2.3, each Spark app generates one Spark event log file. The Spark event log file for a Spark streaming app continues to grow while the app is running. Today a file on WASB has a 50000 block limit, and the default block size is 4 MB. So in default configuration the max file size is 195 GB. However, Azure Storage has increased the max block size to 100 MB, which effectively brought the single file limit to 4.75 TB. For more information, see [Scalability and performance targets for Blob storage](../../storage/blobs/scalability-targets.md).
4141

0 commit comments

Comments
 (0)
Please sign in to comment.