Accounting and reporting depend on auditable, accurate, and complete data, which is not easy to gather in the fragmented landscape of digital assets. Our blog series reveals our journey to creating accounting and reporting software that solves this problem.

In the first article of the series, we explain the challenge of retrieving accurate transaction histories from blockchains.

Key topics covered:

• No centralized authority to provide a statement.
  • The data is technically public, yet there is no reliable source of truth.
• The limitations of explorers & third-party APIs to get on-chain data.
  • Poor indexers that were never built for accounting.
  • Unreliable data and performance of APIs.
• The solution: A purpose-built financial data layer for every chain.

Retrieving your transaction history from your bank is easy. You only have to request a statement from the bank, and a complete list of transactions appears instantly.

Unlike banks, there is no centralized authority on a blockchain. No one, in particular, is liable to provide a statement. Compiling a complete list of transactions from across the different chains, DeFi protocols, and smart contracts is a challenge for any enterprise or institution with crypto on its balance sheet. Gathering accurate and auditable data fit for accounting and reporting is an even harder task for crypto businesses.

Blockchains are public ledgers – technically, everyone has the data

A blockchain is a public ledger where there is a publicly verified record of every transaction that has ever occurred on the network.

Layer-1 chains, which include Bitcoin, Ethereum, Avalanche, and NEAR form the infrastructure for other applications, protocols, and networks to be built on top of them. Each of these blockchains has its own native coin, such as Ether for Ethereum, to pay for network fees like sending cryptocurrency or calling a smart contract.

On top of these chains, developers are constantly building new protocols and applications across a variety of use cases, including finance, gaming, and social media.

All of these transactions are being confirmed on-chain.

Since the record of every transaction is publicly available, the network’s members only have to check the on-chain data to verify a transaction, removing the need for trust between users.

Anyone can view blockchain data using a public block explorer like Etherscan for Ethereum or Blockchain.com for Bitcoin.

Anyone can view the transactions associated with any wallet if they know its public address.

The Paradox – public data but no reliable source of truth

Transparency of on-chain transactions is the technology’s greatest strength. However, assembling a complete transaction history from across this fragmented landscape is a difficult task.

Transaction histories are publicly available through block explorers, but they are not reliable. Often transactions are missing. Especially ‘internal transfers’ that are related to smart contract execution. The completeness and accuracy of the explorer data can also vary greatly from chain to chain. This is because they were not built for enterprise-grade accounting.

Crypto enterprises and institutions require a scalable, auditable solution with internal controls and sanity checks that can ensure accuracy and completeness.

Indexing: Turning block-by-block transaction data into a flexible, queryable database

Blockchains store data blocks in a decentralized network, with each series of blocks chained together. Each block keeps records of all new transactions processed in that block and information from the previous block.

However, this is not useful for accounting. We need to filter and locate and search for specific wallets and transactions associated with a particular entity. The process of turning block-by-block time-ordered transaction data into a database that can be queried on a high-level based on specific traits is called indexing.

You may be familiar with this in your search for information online on Google, Bing, or any other search engine. They are crawling and indexing the web to filter relevant search results. In the same way, an indexer must be built for chains that can parse data in a reliable way for accounting and reporting.

The problem with explorers and third-party indexers is that they were never built for enterprise-grade accounting.

The limitations of explorers & third-party APIs to retrieve on-chain data

There are options on the market, public block explorers, and private data indexing companies offer API access to their databases to pull transaction histories on a wallet level.

These tools have two main flaws which make them unsuitable for a crypto accounting and financial reporting solution. The two major pitfalls are:

• Incomplete and inaccurate data: transactions are often missing or completely inaccurate.
• APIs unsuitable for accounting and reporting: the APIs struggle to return all on-chain data since they were not built with certain use cases in mind.

Incomplete and inaccurate data

Public block explorers fail to capture completely the complex detail of on-chain transactions that change your balance - leaving you with an inaccurate picture of your crypto incomings and outgoings.

For businesses and institutions that have to demonstrate auditability within their reporting, missing or inaccurate data is unacceptable.

Causes of missing transactions include:

• specific transaction categories are not supported, such as internal transfers. These transactions most commonly occur after smart-contract execution.
• transaction code is incorrectly parsed by the third-party tool to read blockchain data.
• transactions are incorrectly reported as failed after chain reorganization, which occurs when two blocks are mined at the same time.

Only a full archival node, which stores the complete transaction history and additional information on each block, can track internal transfers. Many third-party data providers don’t use a full archival node since it is expensive to operate and engineers must monitor it for errors in smart contract execution, which occur often.

Unsuitable APIs for accounting and reporting

Many third-party blockchain data providers expose the data through a paid-for API. None of these APIs are purpose-built for accounting and reporting use cases, meaning that the data selected does not reflect all the data needed by accountants, bookkeepers, and auditors.

After evaluating many of these tools, we decided that they were not suitable to power our accounting and financial reporting. Across the tools we tested, we found the two following limitations with these APIs:

APIs are never truly compatible with all use cases. The data is always organized in a way that favors the use cases that the developers had in mind when designing the API. For example, Blockchain Block Explorer REST API is built for block explorers and the data transferred by this API reflects that use case’s needs.

When using APIs to retrieve blockchain data, the database becomes a black box from which you can only retrieve data. You can’t perform any checks on this data to test its accuracy and completeness.

At Cryptio, we perform checks to spot discrepancies between the on-chain balance and the balance returned to our platform to ensure data accuracy. Many other commercial tools and public block explorers do not carry out discrepancy checks - leaving you with no guarantee of data reliability.

In conclusion, general-purpose indexed data, through explorers or third-party APIs are not suitable for accounting and auditing.

Coming up next in the series Counting Crypto

Stay tuned for the next installment in our series “Counting crypto”. We explain Cryptio’s approach to retrieving accurate and auditable transaction histories from blockchains, with our self-hosted blockchain nodes and purpose-built indexers for crypto accounting and reporting.