accounts/abi/bind: support event filtering in abigen

[karalabe]

Quite a long time ago we landed support for native Go bindings for Ethereum contracts. Back then we figured events will come very soon, but they turned out to require a complete overhaul of our subscription system, filtering system, etc. In short, it never happened.

This PR is fulfilling that promise made long ago, expanding abigen with support for strongly typed, native event filtering for Ethereum contracts from within Go!

Supersedes #15246 which only started work towards event filtering, but did not generate typed Go code. Supersedes #15832 which didn't support filtering at all, just full retrieval.

---

This change is both-way incompatible:

• Bindings generated prior to this version will need to be regenerated due to modifications made to the base layer on top of which abigen contracts are running.
• Bindings generated with the new abigen will require the underlying supporting code (go-ethereum) to be updated too as the new code will depend on newly added features.

Existing client code will not break as no client API changes have been made, only extensions!

---

As a recap, abigen could take a Solidity source file (or ABI spec with an optional EVM bytecode) and generate a Go wrapper out of it. For out original code token.sol, this looked like token.go.

The highlights were automatically generated types and methods to fully interact with the Token contract described in the Solidity file (sample API snippet):

// DeployMyToken deploys a new Ethereum contract, binding an instance of MyToken to it.
DeployMyToken(auth *bind.TransactOpts, backend bind.ContractBackend, initialSupply *big.Int, tokenName string, decimalUnits uint8, tokenSymbol string) (common.Address, *types.Transaction, *MyToken, error)

// BalanceOf is a free data retrieval call binding the contract method 0x70a08231.
//
// Solidity: function balanceOf( address) constant returns(uint256)
func (_MyToken *MyToken) BalanceOf(opts *bind.CallOpts, arg0 common.Address) (*big.Int, error) {

// Transfer is a paid mutator transaction binding the contract method 0xa9059cbb.
//
// Solidity: function transfer(_to address, _value uint256) returns()
func (_MyToken *MyToken) Transfer(_to common.Address, _value *big.Int) (*types.Transaction, error) {

These generated methods could be used to deploy new contracts, bind existing ones, call methods on the contracts to read its state, and initiate state modifying transactions, all from within Go with strongly typed method arguments and return values.

---

This PR expands the generated code with new types and methods to support filtering past contract events, and subscribing to a stream of future events!

For every event present in the contract, abigen will create a Go counterpart. E.g. for the Transfer event in the token contract:

event Transfer(address indexed from, address indexed to, uint256 value);

// MyTokenTransfer represents a Transfer event raised by the MyToken contract.
type MyTokenTransfer struct {
	From  common.Address
	To    common.Address
	Value *big.Int
	Raw   types.Log // Blockchain specific contextual infos
}

For each event, abigen will generate a Filter<event name> method to retrieve past logs. As you can see below, the topics being filtered for are strongly typed Go types, not topic hashes as all other APIs surface. This allows calling code to be meaningfully read and understood without having to guess what some cryptic numbers mean. abigen will generate all the necessary code to convert the user types into topic filter criteria under the hood.

The log filterer method returns an iterator that can be used to iterate over the found logs, each already unpacked from its raw Ethereum RLP encoded form (and topic form) into strongly typed Go structs like MyTokenTransfer listed above.

// FilterTransfer is a free log retrieval operation binding the contract event 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef.
//
// Solidity: event Transfer(from indexed address, to indexed address, value uint256)
func (_MyToken *MyToken) FilterTransfer(opts *bind.FilterOpts, from []common.Address, to []common.Address) (*MyTokenTransferIterator, error) {

Similarly, for each event, abigen will generate a Watch<event name> method to subscribe to future logs and similarly to filtering past events, subscribing to future ones can be done via strongly typed criteria.

The method also takes a sink channel as an argument to deliver newly found contract events on, and returns a subscription which can be used to tear down the watcher constructs. As with past event subscriptions, the events streamed in the user-provided channel are strongly typed like MyTokenTransfer above.

// WatchTransfer is a free log subscription operation binding the contract event 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef.
//
// Solidity: event Transfer(from indexed address, to indexed address, value uint256)
func (_MyToken *MyTokenFilterer) WatchTransfer(opts *bind.WatchOpts, sink chan<- *MyTokenTransfer, from []common.Address, to []common.Address) (event.Subscription, error) {

The full generated code with this PR's abigen is available at token.go.

Caveats

Filtering for past log events returns an iterator for future API stability. The current implementation under the hood runs the entire user filtering in one big go, caches the entire results and then uses the iterator to simulate streaming filtering. This will be replaced eventually, but to avoid breaking the API then, we're enforcing this future mode of operation in user code already now.

Watching for future log events has an optional parameter for specifying the starting block number. This is currently a noop as go-ethereum does not yet support subscribing to past events, but when support lands for it, abigen will naively be able to use it without API changes.

[obscuren]

Long time coming. Well done Peter 👏🏼

[ligi]

Minor typos

[ligi]

-a

[ligi]

flattem -> flatten

[ligi]

"strongly types bound iterator" -> "strongly typed bound iterator"

[gballet]

Question about the API: why not using a single iterator for past and future events?
Also, need some clarification about the disappearance of Indexed, didn't understand that part.

[gballet]

I'm wondering why you want to make a distinction between past and future logs? Both of them could be handled by the same iterator: you could just make the Iter() function take a start parameter that would either say something like FUTURE_ONLY or PAST_PRESENT_AND_FUTURE events. I don't see any case when a user interested in past event would not also be interested in future events. Also, having an API not making the distinction would save the caller code the trouble of having to handle the case when new events arrive between the moment when they called FilterLogs and the time they call WatchLogs

[karalabe]

E.g. When I'm filtering for past events, my code will usually block waiting for the next past event to arrive, until I consumed all past events I filtered for and the subsystem tells me there are no more events coming.

When I'm filtering for future events, my code will not block, rather be event driven, waiting on multiple possible event sources for data to arrive and react to.

---

To be honest, I can't think of an application where I would want to handle past and future events the same way.

E.g. I can filter for past Akasha events to retrieve the stuff the user posted and serve on a restful API, but that needs to stop and return, not wait for arbitrary future events. On the other hand, I can filter for future Akasha events to detect when users post new content and cache them in my API server via IPFS immediately during posting while the user is still online.

The two use cases are wildly different and have different requirements.

[gballet]

To be honest, I can't think of an application where I would want to handle past and future events the same way.

I was thinking that implementing something like tail -f in Unix would be very complicated with this approach. I guess that this is an unlikely use case, then.

[karalabe]

The latter WatchXYZ is the one which supposed to handle that case.

[gballet]

I don't understand why you remove this from the list of fields to be un-marshaled? I understand that it is derived from another source, but what if the field is still present in some client code?

[karalabe]

Indexed only makes sense in the context of an event field (which is inside "inputs"). In the outer ABI context there's no notion of "indexed" afaik.

[gballet]

LGTM

[shoenseiwaso]

Great work, thank you!

Presumably if I want to iterate through all events in the order they are received (as opposed to per-event-type ordering), I would still need to use the FilterLogs()/SubscribeFilterLogs() => select { ... } => UnpackLog() (<== new and supports indexed events now, thank you!) ?

Update: I was able to refactor my code fairly simply by replacing Unpack() with UnpackLog() (and my custom event structs with those generated by abigen).

Would you consider making contract *bind.BoundContract public? It's accessible to me in this case because the generated ABI Go code is in the same package, but it feels more natural if it were public.

Before:

// Indexed events not unpacked correctly
err = TokenABI.Unpack(data, eventName, log.Data)

After:

// Indexed events unpacked correctly
err = Token.TokenCaller.contract.UnpackLog(data, eventName, log)

[dt665m]

I know this is a pretty new PR but is there any documentation or usage examples of watching events? Haven't been able to get it to work. Does this support 'geth --light'? This is what I tried:

var start uint64 = 0
ch := make(chan *TesContractNewDataSet)
opts := &bind.WatchOpts{}
opts.Start = &start
sub, err := contract.WatchNewDataSet(opts, ch)
if err != nil {
	log.Fatal("subscription failed:", err)
}

I then select on 'ch' and 'sub.Err()' but nothing fires. Normal contract calls work so I know my process is able to connect to the geth IPC. Any suggestions?

[writetosalman]

A code example will really help here. Thanks.

[ArtMartiros]

Working example

My event

event YearChanged(uint32 indexed year);

Past events

opt := &bind.FilterOpts{}
s := []uint32{}
past, err := contract.FilterYearChanged(opt, s)
if err != nil {
	log.Fatalf("Failed FilterYearChanged: %v", err)
}
notEmpty := true
for notEmpty {
	notEmpty = past.Next()
	if notEmpty {
		fmt.Println("event log:", past.Event.Year)
	}
}

Listen future events

var blockNumber uint64 = 2900000
s := []uint32{}
ch := make(chan *contract.EventTesterYearChanged)
opts := &bind.WatchOpts{}
opts.Start = &blockNumber
_, err := contract.WatchYearChanged(opts, ch, s)
if err != nil {
	log.Fatalf("Failed WatchYearChanged: %v", err)
}
var newEvent *contract.EventTesterYearChanged = <-ch
fmt.Println(newEvent.Year)

[hossein761]

I am trying to to fetch some past events from a contract as follows. In my example I have used abigen to create bindings for ZIL:

`conn, err := ethclient.Dial("https://mainnet.infura.io/ACCESS_TOKEN")
if err != nil {
log.Fatal("Error connecting to the Ethereum client: ", err)
}

zil, err := bindings.NewZIL(common.HexToAddress("0x05f4a42e251f2d52b8ed15e9fedaacfcef1fad27"), conn)
if err != nil {
	log.Fatal("Failed to instantiate a ZIL contract")
}
opz :=  &bind.CallOpts{}
s, err := zil.Name(opz)
if err != nil {
	log.Error("some error", err)
}

log.Info("name: ", s) // here the name of the contract is correctly printed


   // I want to filter between start and end blocks
end := uint64(4388365)
opts := &bind.FilterOpts{
	Start: 	4288360,
	End: &end,
}
transferEvts, err := zil.FilterTransfer(opts, []common.Address{}, []common.Address{})
if err != nil {
	log.Fatal("Error creating filter on Transfer events", err.Error())
}

notEmpty := true
for notEmpty {
		notEmpty = transferEvts.Next()
		if notEmpty {
			log.Info("Transfer: ", transferEvts.Event.Value)
		} else {
			log.Info("No more events ", transferEvts.Error())
		}
}` 

This follows the example above given by @ArtMartiros . The name of the contract is printed so I am sure the connection works. However, no transaction is printed and transferEvts.Error() is empty.
Can someone please help me with this?

[ArtMartiros]

@hossein761 your code looks correct.

[dimiandre]

How can we handle connection lost ?

actually i have a code like this:

// Setup Watch options (none)
  opts := &bind.WatchOpts{}

  // Create a new streaming channel
  ch := make(chan *contract.TokenTransfer)
  quit := make(chan struct{})

  // attach the channel to a new watcher
  sub, err := contract.WatchTransfer(opts, ch, []common.Address{}, []common.Address{})
  if err != nil {
  	log.Fatalf("Failed Watch Transfers: %v", err)
  }

  for sub != nil {
  	select {
  	case event := <-ch:
  		fmt.Println("My code")
  	
  	case <-quit:
  		fmt.Println("Subscription break")
  		return
  	}
  }

maybe i'm doing something wrong, but quit is never fired. Even if i loose connection to the node.

also would be cool to use the event.Resubscribe function as parameter directly in abigen, what do you think ?

edit: tryed also <- sub.Err(), nothing changes