Log3  - The On-Chain DEBUG Experience

TL;DR — We’ve built a solution to extract commented console.sol logs from on-chain

transactions, allowing better logging.

https://log3.spherex.xyz/

https://github.com/spherex-xyz/log3

‍

Coming from Web2, one of the most invaluable capabilities of a developer is logging.
Although might be heavy on CPU and disk usage, partially activating it for a limited
period of time allows the developer to gain deep insights about how his code operates
and provides the best infrastructure for finding bugs.

All you had to do was defining log lines in your code, set the respective level per log
line and set the minimum log level before execution. As any Web2 developer knows,
informative log and error lines are crucial for debugging.

Moving forward to Web3 we were looking for similar functionality. At spherex we’re
developing highly complexed smart contracts. Debugging & logging capabilities are
crucial. We tried to understand what were the options currently available:

1. Struct logs (aka vmtrace aka Geth Debug Trace) — The most detailed trace out
   there. This is a trace of actual opcodes running on the EVM during transaction
   execution. As in web2 low level debugging, it’s really hard to extract the inner
   state walking through this trace. We were in search of something simpler.
2. Call trace (aka Parity VM Trace) — A (sometimes nested) list of internal calls and
   special events happening during transaction execution. This includes some very
   useful information like function parameters, return values, contracts created
   and destroyed etc. But we were still missing a level of granularity, what happens
   inside the contract while it is being executed?
3. Emitted events (aka logs) — These are actual log lines! One can define an event
   type and emit it with relevant data stemming from the contract inner state. We
   found a few problems using those:

• They are very gas heavy, price that is being imposed on tx.origin
• They’re not robust, the fact that we need to define the event type before
  emitting takes it’s toll. Also, not supported inside static calls.
• They’re for everyone to see. These are mere debug logs for internal use, we
  don’t want it to be indexed by everyone and clutter the blockchain

1. Console.sol (no aka :) ) — This is the closest thing we found to debug logs. Just
   need to write text, attach parameters and voilà, hardhat/foundry will output the
   resolved log lines with ease. But wait, we don’t use HH/forge for actual
   transactions, so how can we catch those log lines on chain? This is indeed the
   major problem with this library, it is not really suitable for post-deployment.

Frustrated with the options in front of us, we asked ourselves what can we do to
gain insights into the inner execution of the transaction — log lines in “production”?

‍

So, with all the options in mind, it made sense to start off with console.sol . We
asked ourselves, what to we need to do to make it “production”-friendly, in the
sense that:

1. Will be easy to integrate
2. Will not incur any gas during execution
3. Switching from “Debug” to “Production” will be easy

We first wanted to have a sense of how widely used console.sol is:

In addition, the amount of console.sol we’ve seen in the verified section on Etherscan,
though not deployed, was huge. It is probably the most common debugging library
in use today.

Of course, it is bad practice to include console logs in deployed contracts, so another
common practice we witnessed was commenting out those lines before deployment:

So as PoC we decided to go with this approach. Leverage the practice of commenting
out console.log lines as way of easily integrating debug logs into deployed contracts.

‍

To start working on utilizing this approach for debug logs, we decided to use foundry
as our dev library. Although we have mixed feelings regarding rust, we feel like
foundry is currently the most feature complete library for our needs. Moreover, is
has the fastest integrated EVM which will help us during the simulation process
explained later. Some bug fixes had to be integrated into foundry for it to work
though [1][2][3][4] .

Given a deployed contract with commented out console.log lines, there are several
steps needed in order to extract the relevant log lines:

1. Fetch sources from Etherscan
2. Uncomment the log lines and add reference to console.sol
3. Recompile sources with uncommented console.log lines and with added
   console.sol source
4. Simulate transaction with overridden contract code

Step 1 is pretty straight forward, it is already well implemented inside foundry.

For step 2 (again, this is a PoC) we’ve used a simple regex replacement:
(\n[ \t]*)//([ \t]*console.log) Will be replaced with $1$2 which will effectively remove
the // sign. This is obviously not a bulletproof method, but it’s good for now.

For step 3 foundry does all the heavy lifting. It stores the fetched sources in a
project-like object that can easily be altered and compiled.

Step 4 is the tricky one. Since we are simulating a mined transaction, we need to
reconstruct the exact state the blockchain was at right before the transaction
started executing. This generally means forking to tx.block-1 and replaying all
block transactions before the current ones. This can sum up to hundreds of
eth_getCode and eth_getStorageAt calls which will prevent the simulation from
finishing in a timely fashion.

Instead, we went with a different approach. There’s a lesser known usage for
debug_traceTransaction involving getting the entire state a transaction requires
to run — prestateTracer . This type of tracer will produce all the code, balances,
nonce and storage slots a certain transaction requires to execute properly, all in
one single RPC call. This approach reduces the simulation time dramatically.

Another aspect is the gas limits set in the original transaction. Since we’re altering
the code, the gas consumption of the simulated transaction will probably increase.
It makes sense to increase the limits before actually simulating the transaction.

And the end result — for transaction 0xa41a5e85874c305b2cdc1c0f529d2025ae41b5ed865452999ace0c385cc9fc4e
using the contract 0x60FB45483d9E48dFC602f49E094024F8DA292416:

‍

coinbase 0x1aD91ee08f21bE3dE0BA2ba6918E714dA6B45836
difficulty 7952037238692364445226370365715508883266782329560800400971864222
4436144452329
_nonces 6414
0x409d634b2f86a76bc9706c74fe5e9dc9081bd8076327ca22af1337b71846b819
2585
7846

‍

Some Caveats using this tool

• It is still a PoC, don’t fully trust the results
• Uncommenting the log lines is flaky (e.g. multiline comments)
• Contracts with immutables are currently not supported (since the immutable values are missing from our version of the deployed bytecode)
• We’re playing with the simulated gas, don’t trust the gas results of the
  simulation
• Since some blockchains use some variations in gas profiles that foundry can’t yet simulate, only mainnet is currently supported

‍