Ubuntu build seems to be broken.

[thiagoveloso]

Thanks for the package, it looks really promising along with bigrquery. Under Ubuntu 20.04, however, I am unable to install it.

First of all, dependencies. My initial attempt was the easier route:
sudo apt-get install -y libgrpc++-dev libprotobuf-dev protobuf-compiler-grpc

which resulted in a protobuf version that is a bit old:

$ protoc --version
libprotoc 3.6.1

For this scenario, this is the package instalation error I got:

> install.packages("bigrquerystorage")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/bigrquerystorage_1.2.1.tar.gz'
Content type 'application/x-gzip' length 57448 bytes (56 KB)
==================================================
downloaded 56 KB

* installing *source* package ‘bigrquerystorage’ ...
** package ‘bigrquerystorage’ successfully unpacked and MD5 sums checked
** using staged installation
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-pthread
Using PKG_LIBS=-lprotobuf -pthread -lgrpc++
Found C++17 compiler: g++ -std=gnu++17 -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2
Using libprotoc 3.6.1 from /usr/bin/protoc
Using /usr/bin/grpc_cpp_plugin
Missing value for flag: --experimental_allow_proto3_optional
Missing value for flag: --experimental_allow_proto3_optional
Error: failed to compile proto files.
ERROR: configuration failed for package ‘bigrquerystorage’
* removing ‘/usr/local/lib/R/site-library/bigrquerystorage’
Warning in install.packages :
  installation of package ‘bigrquerystorage’ had non-zero exit status

The downloaded source packages are in
	‘/tmp/RtmpkVJekT/downloaded_packages’

Then, I uninstalled the system-provided dependencies and decided to compile a recent version of gRPC from source, which also builds protobuf. This is how I did it:

git clone -b v1.67.0 https://github.com/grpc/grpc
cd grpc
git submodule update --init
mkdir -p cmake/build
cd cmake/build
cmake -j4 ../..
sudo make -j4 install

But still I got an error (different one, though):

> install.packages("bigrquerystorage")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/bigrquerystorage_1.2.1.tar.gz'
Content type 'application/x-gzip' length 57448 bytes (56 KB)
==================================================
downloaded 56 KB

* installing *source* package ‘bigrquerystorage’ ...
** package ‘bigrquerystorage’ successfully unpacked and MD5 sums checked
** using staged installation
Using PKG_CFLAGS=
Using PKG_LIBS=-lgrpc++ -lgrpc -lprotobuf
Found C++17 compiler: g++ -std=gnu++17 -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2
Using libprotoc 27.2 from /usr/local/bin/protoc
Using /usr/local/bin/grpc_cpp_plugin
** libs
using C++ compiler: ‘g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0’
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/field_behavior.pb.cc -o google/api/field_behavior.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/http.pb.cc -o google/api/http.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/launch_stage.pb.cc -o google/api/launch_stage.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/resource.pb.cc -o google/api/resource.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/arrow.pb.cc -o google/cloud/bigquery/storage/v1/arrow.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/avro.pb.cc -o google/cloud/bigquery/storage/v1/avro.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/table.pb.cc -o google/cloud/bigquery/storage/v1/table.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/annotations.pb.cc -o google/api/annotations.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/api/client.pb.cc -o google/api/client.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/protobuf.pb.cc -o google/cloud/bigquery/storage/v1/protobuf.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/stream.pb.cc -o google/cloud/bigquery/storage/v1/stream.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/rpc/status.pb.cc -o google/rpc/status.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/storage.pb.cc -o google/cloud/bigquery/storage/v1/storage.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c google/cloud/bigquery/storage/v1/storage.grpc.pb.cc -o google/cloud/bigquery/storage/v1/storage.grpc.pb.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I.  -I'/usr/local/lib/R/site-library/Rcpp/include'    -fvisibility=hidden -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-thBIeW/r-base-4.4.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2   -c bqs.cpp -o bqs.o
bqs.cpp:9:33: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
    9 | # pragma GCC diagnostic ignored "-Winconsistent-missing-override"
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bqs.cpp:20:22: error: variable or field ‘bqs_default_log’ declared void
   20 | void bqs_default_log(gpr_log_func_args* args) {
      |                      ^~~~~~~~~~~~~~~~~
bqs.cpp:20:22: error: ‘gpr_log_func_args’ was not declared in this scope
bqs.cpp:20:41: error: ‘args’ was not declared in this scope
   20 | void bqs_default_log(gpr_log_func_args* args) {
      |                                         ^~~~
bqs.cpp: In function ‘void bqs_set_log_verbosity(int)’:
bqs.cpp:32:3: error: ‘gpr_set_log_verbosity’ was not declared in this scope; did you mean ‘bqs_set_log_verbosity’?
   32 |   gpr_set_log_verbosity(static_cast<gpr_log_severity>(severity));
      |   ^~~~~~~~~~~~~~~~~~~~~
      |   bqs_set_log_verbosity
bqs.cpp: In function ‘void bqs_init_logger()’:
bqs.cpp:38:24: error: ‘bqs_default_log’ was not declared in this scope
   38 |   gpr_set_log_function(bqs_default_log);
      |                        ^~~~~~~~~~~~~~~
bqs.cpp:38:3: error: ‘gpr_set_log_function’ was not declared in this scope
   38 |   gpr_set_log_function(bqs_default_log);
      |   ^~~~~~~~~~~~~~~~~~~~
make: *** [/usr/lib/R/etc/Makeconf:204: bqs.o] Error 1
ERROR: compilation failed for package ‘bigrquerystorage’
* removing ‘/usr/local/lib/R/site-library/bigrquerystorage’
Warning in install.packages :
  installation of package ‘bigrquerystorage’ had non-zero exit status

What am I missing? Any advice to fix the error would be much appreciated!

[meztez]

uninstall the system package? try an earlier version of grpc? I saw some work around replacing the log.cc that this package uses. It might be that? grpc/proposal#425

[meztez]

I was not able to install on focal using the current toolchain. I won't be supporting focal, sorry.

[thiagoveloso]

Just for future reference, I managed to install bigrquerystorage on a rather old (Focal - 20.04) Ubuntu machine. The key step was downgrading gRPC to an older version as well - in my case v1.46.0. Here are the detailed build instructions:

git clone -b v1.46.0 https://github.com/grpc/grpc
cd grpc
git submodule update --init --recursive
mkdir -p cmake/build
cd cmake/build

cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_STANDARD=17 \
  -DCMAKE_CXX_FLAGS="-std=c++17" \
  -DCMAKE_INSTALL_PREFIX=/usr/local \
  -DgRPC_INSTALL=ON \
  -DgRPC_BUILD_TESTS=OFF \
  -DgRPC_ABSL_PROVIDER=module \
  -DgRPC_CARES_PROVIDER=module \
  -DgRPC_PROTOBUF_PROVIDER=module \
  -DgRPC_RE2_PROVIDER=module \
  -DgRPC_SSL_PROVIDER=package \
  -DgRPC_ZLIB_PROVIDER=package \
  -DABSL_ENABLE_INSTALL=ON \
  -Dprotobuf_INSTALL=ON \
  -Dprotobuf_BUILD_TESTS=OFF \
  -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
  ../..

make -j$(nproc)
make install

Queries are now much snappier in bigrquery using the arrow API. Definitely worth installing bigrquerystorage!