1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
|
# Log Compression instead of Chatty in Android S
## The problem
* Log buffer space is precious, but suffers from the tragedy of the commons
* Log spam fills the buffers making them less useful in logcat/bugreports
* “Spam” is often in the eye of the beholder: which messages are important depends on what you’re trying to debug
## The idea
* Chatty isn’t helping as much as we’d hoped, and is surprisingly expensive
* Compress logs to make more efficient use of the buffer
* Address the root cause of log spam at its source:
* Do not hide log spam at runtime, which de-incentivize fixes
* Add presubmit coverage similar to SELinux violations to keep log spam down
---
## Chatty in Theory
* Delete messages classified as spam to extend the range of logs from other sources
* “Spam” defined as:
* Logs from UIDs whose logs consume over 12.5% of a log buffer
* Back-to-back exact duplicate messages
## Chatty in Practice
* Developer confusion about missing and de-duplicated logs
* Lowered incentive to fix the root cause of bad logging behavior
* High CPU overhead
* Memory usage greatly exceeds configured buffer size
* Only marginal increase in log range
---
## Log Compression in Theory
* Store many more logs in the same log buffer size => better for diagnosis
* Memory usage stays below configured log size => better system health
* No gaps in logs, no de-duplicated logs => no developer confusion
* No hiding bad behavior => increased accountability/incentive to fix root causes
## Log Compression Preliminary Results
* Captured 2, 5 day periods of full time personal usage of Pixel 4 and replayed the logs offline
* Compression vs Chatty:
* **3.5x more log messages on average**
* **50% less CPU usage**
* **50% less memory usage**
---
## Log Messages in 1MB
* The number of log messages still available in logcat after ‘Message Count’ messages have been logged to a 1MB log buffer
* Note: ‘Simple’ is the Chatty code without log spam detection and without de-duplication.
---
## CPU Time
* Total CPU time on ARM64 (Walleye) and 32bit x86 (Cuttlefish)
* X axis represents different log buffer size configurations.
* Chatty uses significantly more CPU time at 1MB (the default Pixel configuration)
* Chatty scales poorly with increased log buffer sizes
* Note: “simple” isn’t “compression without actually compressing”, it’s “chatty without doing the chatty elimination”, which is why “simple” is more expensive than “compression” on walleye.
---
## Memory Usage
* The memory used by ‘Message Count’ messages, on both Walleye and Cuttlefish
* Note: Chatty does not consider the metadata (UID, PID, timestamp, etc) in its calculation of log buffer size, so a 1MB log buffer will consume more than 1MB. Note that there are 8 log buffers, 5 of which are typically filled.
|
