Is it worth downloading artifacts from a remote build cache node on a slow internet connection?

值得在缓慢的Internet连接上从远程构建缓存节点下载工件吗？

This question is becoming ever more important as we all switch from our blazing fast office internet connections to our home internet connections.

随着我们都从快速的办公室高速互联网连接切换到家庭互联网连接，这个问题变得越来越重要。

This is part 1 of a 3 part series. Part 2 is now published.

这是3部分系列的第1部分。 第2部分现已发布。

下载单个任务 (Downloading a Single Task)

How do we know if a remote build cache is worth it? Let’s take a simple Gradle task and calculate whether having a remote build cache is worth it.

我们如何知道远程构建缓存是否值得？ 让我们执行一个简单的Gradle任务，并计算是否值得拥有一个远程构建缓存。

If a task takes 10 seconds to execute and generates 1 MB of output, you would need an internet connection faster than 0.1 MB/s in order to save time. An internet connection of 1MB/s would mean that you could download the output of the task in 1 second, saving you 9 seconds of execution time.

如果一项任务需要10秒才能执行并产生1 MB的输出，则您需要Internet连接的速度高于0.1 MB / s才能节省时间。 1MB / s的互联网连接意味着您可以在1秒内下载任务的输出，从而节省9秒钟的执行时间。

计算实际Gradle任务 (Calculation for a real Gradle task)

To do this for a Gradle task, you can print its outputs by adding the following to your build.gradle file:

要对Gradle任务执行此操作，可以通过将以下内容添加到build.gradle文件中来打印其输出：

tasks.named("compileDebugKotlin").configure {
  outputs.files.forEach {
    println(it)
  }
}

For the KotlinCompile task, the outputs are placed in that module’s build/tmp/kotlin-classes directory.

对于KotlinCompile任务，将输出放置在该模块的build/tmp/kotlin-classes目录中。

$ du -sh libs/analytics/build/tmp/kotlin-classes/
 136K

This KotlinCompile task produces 136KB of output and takes about 5 seconds to run on my computer. This would require a build cache connection of 0.136MB/5s = .0272 MB/s in order to be able to download the task faster than it would take to run locally.

此KotlinCompile任务产生136KB的输出，大约需要5秒钟才能在我的计算机上运行。 这将需要0.136MB / 5s = .0272 MB / s的构建缓存连接，以便能够比在本地运行更快地下载任务。

The download size is actually smaller than 136KB because Gradle compresses artifacts before placing them in the build cache. This speeds up download time but adds decompression time. We’ll dive in to this later.

下载大小实际上小于136KB，因为Gradle在将工件放入构建缓存之前先对其进行压缩 。 这样可以加快下载时间，但会增加解压缩时间。 我们稍后将深入探讨。

让我们检查一下构建扫描 (Let’s check a build scan)

A build scan will tell us exactly how much time we saved by reusing the outputs of the task from the build cache. There is one caveat, it uses the amount of time that it took to run the task on a different machine when the remote build cache was seeded. In this case, the build cache was seeded from a significantly slower machine. This task only takes 5 seconds to run on my machine, but took 15 seconds when seeding the cache. To get a more accurate estimate, we need to estimate the minimum connection speed based on the local machines power.

通过重用构建缓存中的任务输出，构建扫描将确切告诉我们节省了多少时间。 需要注意的是， 它使用了当远程构建缓存被植入时在另一台机器上运行任务所花费的时间 。 在这种情况下，构建缓存是从速度较慢的计算机中播种的。 在我的计算机上运行此任务仅需5秒钟，而为缓存添加种子时仅需15秒钟。 为了获得更准确的估计，我们需要根据本地计算机的功率来估计最小连接速度。

计算并行计算 (Calculating parallel computation)

Machines these days can perform multiple Gradle tasks at once. To perform the same calculation for two tasks run in parallel, we just take the length of the overall build, and divide it by sum of the two task’s outputs.

这些天的机器可以一次执行多个Gradle任务。 为了对并行运行的两个任务执行相同的计算，我们只需要计算整个构建的长度，然后将其除以两个任务的输出之和即可。

If two 10 second tasks run in parallel, the length of the overall build would still be 10 seconds. If both tasks produce 1MB of output, the sum of the two task’s outputs would be 2MB.

如果两个10秒的任务并行运行，则整个构建的长度仍为10秒。 如果两个任务都产生1MB的输出，则两个任务的输出之和将为2MB。

2MB/10s = 0.2MB/s minimum connection speed

2MB / 10s = 0.2MB / s最小连接速度

The more tasks our local machine can run in parallel, the faster our connection to the build cache node needs to be!

我们的本地计算机可以并行运行的任务越多，我们与构建缓存节点的连接就需要越快！

压缩 (Compression)

Gradle compresses artifacts before storing them in the build cache. In the case of the above KotlinCompile task, the compressed artifacts are only 20KB instead of 136KB. This means our minimum connection speed to make the build cache worth it is much lower: 0.02MB / 5 seconds = 0.004 MB/s. This assumes the artifact decompression time is insignificant.To calculate the size of the compressed artifact, we need to know the build cache key. We find this in the build scan by clicking on the task.

Gradle先压缩工件，然后再将它们存储在构建缓存中。 在上述KotlinCompile任务的情况下，压缩的工件只有20KB，而不是136KB。 这意味着使构建缓存值得使用的最低连接速度要低得多：0.02MB / 5秒= 0.004 MB / s。 这是假设工件解压缩时间不重要的。要计算压缩工件的大小，我们需要知道构建缓存键。 通过单击任务，我们可以在构建扫描中找到它。

The build cache stores artifacts by default in the ~/.gradle/caches/build-cache-1 directory where the name of the file is the build cache key, for example:

默认情况下，构建缓存将工件存储在~/.gradle/caches/build-cache-1目录中，其中文件名是构建缓存键，例如：

du -h ~/.gradle/caches/build-cache-1/b59e646b297605c55f49a251bdd7814

计算典型构建 (Calculating a typical build)

We want to know if the build cache is worth it in a real scenario like assembling our app. To make this calculation, we would divide the amount of time a build would take by the total amount of compressed artifacts that the build produces. For the build time, we should only include the execution phase of the Gradle build. We exclude configuration time since tasks are neither executed nor downloaded during that time.

我们想知道在实际情况下(例如组装我们的应用)构建缓存是否值得。 为了进行此计算，我们将构建花费的时间除以该构建产生的压缩工件的总量。 对于构建时间，我们应该只包括Gradle构建的执行阶段。 我们排除了配置时间，因为在此期间既不执行也不下载任务 。

We run into a bit of a problem when trying to find the total size of all the compressed build cache artifacts. A Gradle build may run hundreds if not thousands of tasks. There is no public API to get the build cache key for a Gradle task. The only way to find the cache key is to manually look at a build scan. If we had to do that for 1000 tasks, we would be here all day.

尝试查找所有压缩的构建缓存工件的总大小时，我们遇到了一个问题。 Gradle构建可能会运行数百个甚至数千个任务。 没有公共API获取Gradle任务的构建缓存密钥。 查找缓存键的唯一方法是手动查看构建扫描。 如果我们必须执行1000个任务，那么我们将整天待在这里。

In order to provide a real estimate, we will dive in to the internals of Gradle in part 2 of this post.

为了提供真实的估计，我们将在本文的第二部分中深入探讨Gradle的内部。

If you want to run an automatic estimation in your project, apply the Gradle Doctor plugin. Then simply run any gradle task with -PbenchmarkRemoteCache.

如果要在项目中运行自动估算，请应用Gradle Doctor插件 。 然后，只需使用-PbenchmarkRemoteCache运行任何gradle任务-PbenchmarkRemoteCache 。

./gradlew :app:assembleDebug -PbenchmarkRemoteCache

You’ll get a nice report like the one below:

您会收到一份不错的报告，如下所示：

=================Gradle Doctor Prescriptions========================
= Remote Build Cache Benchmark Report =
Executed tasks created compressed artifacts of size 159,93 MB
Total Task execution time was 208,85 sIn order for a remote build cache to save you time, you would need an internet connection to your node of at least 0,77 MB/s.
Check a build scan to see your connection speed to the build cache node.
Build cache node throughput may be different than your internet connection speed.A 1 MB/s connection would save you 48,92 s.                                      A 2 MB/s connection would save you 128,88 s.                
A 10 MB/s connection would save you 192,86 s.
                                                                                                Note: This is an estimate. Real world performance may vary. This estimate does not take in to account time spent decompressing cached artifacts or roundtrip communication time to the cache node.   ====================================================================

Let me know how it works for you and your workload. Is using the remote build cache worth it?

让我知道它如何为您和您的工作量。 使用远程构建缓存值得吗？

Thank you to Júlio Zynger for proofreading this post.

感谢JúlioZynger对本文进行校对。