Skip to content

Commit a6eff39

Browse files
scelephilwo
authored andcommitted
[1/3] Bump grpc to 1.33.1 to fix corruption when downloading CAS blobs
Part 1: add v1.33.1 version to third_party/grpc Note: partly switches to v1.33.1 too as not all bits are versioned and some of unversioned bits are used from other third_party targets grpc-java versions 1.27 through 1.32 had a bug where messages could arrive after the call was reported clsoed. In the case of bazel, this meant that in GrpcCacheClient, onNext could be called after onError. This leads to offset bookkeeping getting out of sync, and corrupts the CAS blob download. bazelbuild#12927
1 parent b774992 commit a6eff39

13 files changed

+139
-17
lines changed

third_party/grpc/BUILD

+9-9
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ load("//tools/distributions:distribution_rules.bzl", "distrib_java_import", "dis
1818

1919
licenses(["notice"]) # Apache v2
2020

21-
exports_files(["grpc_1.32.0.patch"])
21+
exports_files(["grpc_1.32.0.patch", "grpc_1.33.1.patch"])
2222

2323
package(default_visibility = ["//visibility:public"])
2424

@@ -32,14 +32,14 @@ filegroup(
3232
distrib_jar_filegroup(
3333
name = "bootstrap-grpc-jars",
3434
srcs = [
35-
"grpc-api-1.32.2.jar",
36-
"grpc-auth-1.32.2.jar",
37-
"grpc-context-1.32.2.jar",
38-
"grpc-core-1.32.2.jar",
39-
"grpc-netty-1.32.2.jar",
40-
"grpc-protobuf-1.32.2.jar",
41-
"grpc-protobuf-lite-1.32.2.jar",
42-
"grpc-stub-1.32.2.jar",
35+
"grpc-api-1.33.1.jar",
36+
"grpc-auth-1.33.1.jar",
37+
"grpc-context-1.33.1.jar",
38+
"grpc-core-1.33.1.jar",
39+
"grpc-netty-1.33.1.jar",
40+
"grpc-protobuf-1.33.1.jar",
41+
"grpc-protobuf-lite-1.33.1.jar",
42+
"grpc-stub-1.33.1.jar",
4343
],
4444
enable_distributions = ["debian"],
4545
)

third_party/grpc/README.bazel.md

+6-6
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,30 @@
11
# How to update the C++ sources of gRPC:
22

33
1. Update the gRPC definitions in WORKSPACE file, currently we use
4-
https://github.com/grpc/grpc/archive/v1.32.2.tar.gz
4+
https://github.com/grpc/grpc/archive/v1.33.1.tar.gz
55
2. Update the gRPC patch file if necessary, it mostly helps avoid unnecessary dependencies.
66
3. Update third_party/grpc/BUILD to redirect targets to @com_github_grpc_grpc if necessary.
77

88
# How to update the BUILD/bzl sources of gRPC:
99

1010
1. `git clone http://github.com/grpc/grpc.git` in a convenient directory
11-
2. `git checkout <tag>` (current is `v1.32.0`, commithash `414bb8322d`)
11+
2. `git checkout <tag>` (current is `v1.33.1`, commithash `054ff69350`)
1212
3. `mkdir -p third_party/grpc/bazel`
1313
4. `cp <gRPC git tree>/bazel/{BUILD,cc_grpc_library.bzl,generate_cc.bzl,protobuf.bzl} third_party/grpc/bazel`
1414
5. In the `third_party/grpc` directory, apply local patches:
15-
`patch -p3 < bazel_1.32.0.patch`
15+
`patch -p3 < bazel_1.33.1.patch`
1616

1717
# How to update the Java plugin:
1818

19-
1. Checkout tag `v1.32.2` from https://github.com/grpc/grpc-java
19+
1. Checkout tag `v1.33.1` from https://github.com/grpc/grpc-java
2020
2. `cp -R <grpc-java git tree>/compiler/src/java_plugin third_party/grpc/compiler/src`
2121

2222
# How to update the Java code:
2323

24-
Download the necessary jars at version `1.32.2` from maven central.
24+
Download the necessary jars at version `1.33.1` from maven central.
2525

2626
# Submitting the change needs 3 pull requests
2727

2828
1. Update third_party/grpc to include files from new version
29-
2. Switch WORKSPACE, scripts/bootstrap/compile.sh and any other references to new version
29+
2. Switch distdir_deps.bzl, scripts/bootstrap/compile.sh and any other references to new version
3030
3. Remove older version from third_party/grpc
File renamed without changes.

third_party/grpc/compiler/src/java_plugin/cpp/java_generator.h

+2-2
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,13 @@ class LogHelper {
3838
}
3939
};
4040

41-
// Abort the program after logging the mesage if the given condition is not
41+
// Abort the program after logging the message if the given condition is not
4242
// true. Otherwise, do nothing.
4343
#define GRPC_CODEGEN_CHECK(x) !(x) && LogHelper(&std::cerr).get_os() \
4444
<< "CHECK FAILED: " << __FILE__ << ":" \
4545
<< __LINE__ << ": "
4646

47-
// Abort the program after logging the mesage.
47+
// Abort the program after logging the message.
4848
#define GRPC_CODEGEN_FAIL GRPC_CODEGEN_CHECK(false)
4949

5050
namespace java_grpc_generator {

third_party/grpc/grpc-api-1.33.1.jar

225 KB
Binary file not shown.

third_party/grpc/grpc-auth-1.33.1.jar

14.2 KB
Binary file not shown.
29.9 KB
Binary file not shown.

third_party/grpc/grpc-core-1.33.1.jar

631 KB
Binary file not shown.
240 KB
Binary file not shown.
5.06 KB
Binary file not shown.
7.45 KB
Binary file not shown.

third_party/grpc/grpc-stub-1.33.1.jar

48.6 KB
Binary file not shown.

third_party/grpc/grpc_1.33.1.patch

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
commit bb0d04663c7dc6c0096f8717cb4ec26330a5ae40
2+
Author: Yun Peng <[email protected]>
3+
Date: Wed Jun 3 15:35:31 2020 +0200
4+
5+
Patch grpc v1.26.0 for Bazel build
6+
7+
- Avoid loading dependencies that're not needed for the gRPC C++
8+
libraries
9+
- Add bazel mirror URL for upb and cares
10+
- Redirect zlib to @//third_party/zlib
11+
- Add darwin_arm64 and darwin_arm64e config settings to allow
12+
building for Apple Silicon
13+
14+
diff --git a/bazel/grpc_build_system.bzl b/bazel/grpc_build_system.bzl
15+
index 7bb6b8bdb9..7644107b70 100644
16+
--- a/bazel/grpc_build_system.bzl
17+
+++ b/bazel/grpc_build_system.bzl
18+
@@ -25,7 +25,7 @@
19+
20+
load("//bazel:cc_grpc_library.bzl", "cc_grpc_library")
21+
load("@upb//bazel:upb_proto_library.bzl", "upb_proto_library")
22+
-load("@build_bazel_rules_apple//apple:ios.bzl", "ios_unit_test")
23+
+# load("@build_bazel_rules_apple//apple:ios.bzl", "ios_unit_test")
24+
25+
# The set of pollers to test against if a test exercises polling
26+
POLLERS = ["epollex", "epoll1", "poll"]
27+
@@ -181,13 +181,13 @@ def ios_cc_test(
28+
testonly = 1,
29+
)
30+
ios_test_deps = [ios_test_adapter, ":" + test_lib_ios]
31+
- ios_unit_test(
32+
- name = name + "_on_ios",
33+
- size = kwargs.get("size"),
34+
- tags = ios_tags,
35+
- minimum_os_version = "9.0",
36+
- deps = ios_test_deps,
37+
- )
38+
+ # ios_unit_test(
39+
+ # name = name + "_on_ios",
40+
+ # size = kwargs.get("size"),
41+
+ # tags = ios_tags,
42+
+ # minimum_os_version = "9.0",
43+
+ # deps = ios_test_deps,
44+
+ # )
45+
46+
def grpc_cc_test(name, srcs = [], deps = [], external_deps = [], args = [], data = [], uses_polling = True, language = "C++", size = "medium", timeout = None, tags = [], exec_compatible_with = [], exec_properties = {}, shard_count = None, flaky = None):
47+
copts = if_mac(["-DGRPC_CFSTREAM"])
48+
diff --git a/bazel/grpc_deps.bzl b/bazel/grpc_deps.bzl
49+
index 09fcad95a2..9b737e5deb 100644
50+
--- a/bazel/grpc_deps.bzl
51+
+++ b/bazel/grpc_deps.bzl
52+
@@ -33,7 +33,7 @@ def grpc_deps():
53+
54+
native.bind(
55+
name = "madler_zlib",
56+
- actual = "@zlib//:zlib",
57+
+ actual = "@//third_party/zlib",
58+
)
59+
60+
native.bind(
61+
diff --git a/bazel/grpc_extra_deps.bzl b/bazel/grpc_extra_deps.bzl
62+
index 4c1dfad2e8..f63c54ddef 100644
63+
--- a/bazel/grpc_extra_deps.bzl
64+
+++ b/bazel/grpc_extra_deps.bzl
65+
@@ -1,11 +1,6 @@
66+
"""Loads the dependencies necessary for the external repositories defined in grpc_deps.bzl."""
67+
68+
-load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
69+
load("@upb//bazel:workspace_deps.bzl", "upb_deps")
70+
-load("@envoy_api//bazel:repositories.bzl", "api_dependencies")
71+
-load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
72+
-load("@build_bazel_rules_apple//apple:repositories.bzl", "apple_rules_dependencies")
73+
-load("@build_bazel_apple_support//lib:repositories.bzl", "apple_support_dependencies")
74+
75+
def grpc_extra_deps():
76+
"""Loads the extra dependencies.
77+
@@ -26,15 +21,5 @@ def grpc_extra_deps():
78+
grpc_extra_deps()
79+
```
80+
"""
81+
- protobuf_deps()
82+
-
83+
upb_deps()
84+
85+
- api_dependencies()
86+
-
87+
- go_rules_dependencies()
88+
- go_register_toolchains()
89+
-
90+
- apple_rules_dependencies()
91+
-
92+
- apple_support_dependencies()
93+
diff --git a/third_party/cares/cares.BUILD b/third_party/cares/cares.BUILD
94+
index c047f0c515..7c24fbc617 100644
95+
--- a/third_party/cares/cares.BUILD
96+
+++ b/third_party/cares/cares.BUILD
97+
@@ -10,6 +10,16 @@ config_setting(
98+
values = {"cpu": "darwin_x86_64"},
99+
)
100+
101+
+config_setting(
102+
+ name = "darwin_arm64",
103+
+ values = {"cpu": "darwin_arm64"},
104+
+)
105+
+
106+
+config_setting(
107+
+ name = "darwin_arm64e",
108+
+ values = {"cpu": "darwin_arm64e"},
109+
+)
110+
+
111+
config_setting(
112+
name = "windows",
113+
values = {"cpu": "x64_windows"},
114+
@@ -99,6 +109,8 @@ copy_file(
115+
":watchos_arm64_32": "@com_github_grpc_grpc//third_party/cares:config_darwin/ares_config.h",
116+
":darwin": "@com_github_grpc_grpc//third_party/cares:config_darwin/ares_config.h",
117+
":darwin_x86_64": "@com_github_grpc_grpc//third_party/cares:config_darwin/ares_config.h",
118+
+ ":darwin_arm64": "@com_github_grpc_grpc//third_party/cares:config_darwin/ares_config.h",
119+
+ ":darwin_arm64e": "@com_github_grpc_grpc//third_party/cares:config_darwin/ares_config.h",
120+
":windows": "@com_github_grpc_grpc//third_party/cares:config_windows/ares_config.h",
121+
":android": "@com_github_grpc_grpc//third_party/cares:config_android/ares_config.h",
122+
"//conditions:default": "@com_github_grpc_grpc//third_party/cares:config_linux/ares_config.h",

0 commit comments

Comments
 (0)