Apps: txn.Access list for access to more resources

[jannotti]

Summary

Today, app developers find it frustrating that they can only list 8 resources in a transaction. This number is artificially low because there are rules that allow access to many more than 8 items when, for example, 4 account and 4 apps are listed.

This PR introduced a single unified Access field on app calls, which contains all accounts, apps, asas, and boxes that the transaction can touch. Because no extra rules allow access to extra resources, we can expand the allowable size of such a list. 32 seems likely. (edit: currently thinking 16, but also upping the box quota per reference to 2k)

This will probably increase performance, since it will now be reasonable to perform perfect prefetching of all resources an app call might touch.

This PR also augments goal to create these transactions if the --access flag is used. It still needs e2e_subs tests that exercise it. changing back to draft until I write those

This PR does not actually implement the improved pre-fetching. Should it?

Test Plan

[joe-p]

This shouldn't need a new AVM version, correct? Since AVM9 we've had group resource sharing, which means an app has no way of knowing what resources are available. Although I suppose it could check if it's an outer call and check the rest of the group, but seems highly improbable to account for all the sharing rules even if an app wanted to do that.

[jannotti]

I think you're asking "will only new programs, AVM v12, be able to use this?"

I think it should be ok to let old programs use this, including letting these resources be accessed by other programs in the same group that are a low version. I'm pretty sure we made the same decision with resource pooling. After the consensus upgrade, programs suddenly got access to things they didn't "know" they had access to by looking into the arrays.

I'll have to confirm whether we put in any limitations. For example, I think it would be unsafe to let v3 (!) programs see extra ASAs.

[codecov]

Codecov Report

Attention: Patch coverage is 61.25000% with 310 lines in your changes missing coverage. Please review.

Project coverage is 50.64%. Comparing base (9adaf55) to head (b0e1211).
Report is 10 commits behind head on master.

Files with missing lines	Patch %	Lines
cmd/goal/application.go	29.19%	113 Missing and 1 partial ⚠️
data/basics/testing/nearzero.go	69.29%	35 Missing and 4 partials ⚠️
data/transactions/application.go	81.43%	26 Missing and 5 partials ⚠️
libgoal/transactions.go	80.83%	23 Missing ⚠️
util/fn.go	0.00%	23 Missing ⚠️
shared/pingpong/pingpong.go	0.00%	18 Missing ⚠️
cmd/goal/interact.go	0.00%	14 Missing ⚠️
cmd/goal/asset.go	0.00%	12 Missing ⚠️
daemon/algod/api/server/v2/utils.go	0.00%	12 Missing ⚠️
cmd/goal/clerk.go	0.00%	6 Missing ⚠️
... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6286      +/-   ##
==========================================
+ Coverage   49.81%   50.64%   +0.82%     
==========================================
  Files         356      656     +300     
  Lines       64417   111052   +46635     
==========================================
+ Hits        32091    56240   +24149     
- Misses      30970    51941   +20971     
- Partials     1356     2871    +1515     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joe-p]

Right that's what I would think as well. The main reason I ask is because if we don't require a specific AVM version existing applications, even if immutable, can leverage this feature by simply updating their client-side code.

[jannotti]

Right that's what I would think as well. The main reason I ask is because if we don't require a specific AVM version existing applications, even if immutable, can leverage this feature by simply updating their client-side code.

Having worked on it some more:

Yes, any transaction can use tx.Access instead of the existing arrays, even if the transaction is an app call to an old app. Old apps that use "slots" with opcodes will index into tx.Access instead of the respective array. That element of tx.Access has to be the proper type. So int 3; int 4; asset_holding_get will expect an account in tx.Access[3-1] and an asset in tx.Access[4-1]. (The -1 is because tx.Access is always 1-based so that 0 can mean Sender or current app for address and apps.)

As for sharing, we only enabled sharing for v9 programs and higher, so I made that true here as well. Your v9 or higher programs will be able to see the resources made available from other transactions, whether those transactions made the resources available with the "old-style" foreign arrays or with tx.Access.

Like tx.Boxes, I do not plan on making tx.Access explicitly visible to apps, there will not be txna Access 3. And inner transactions will not be able to populate tx.Access for their inners. The first limitation is to prevent apps from the bad practice of passing arguments in tx.Access. That should be done explicitly, so that availability can be handled in various ways (maybe another transaction has the resource in tx.Access, for example). The second limitation is to reduce complexity - generally you shouldn't need to pass tx.Access because of resource sharing.

[algorandskiy]

1. Left some comments
2. Empty box ref in Access gaining extra reads budged should be somehow better documented

[Copilot]

Pull Request Overview

This PR introduces a unified Access field for app calls to replace the multiple resource reference arrays (accounts, assets, apps, boxes) with a single list that can support up to 16-32 resources instead of the current 8-resource limit.

Key changes:

• Adds new Access field to ApplicationCallTxnFields containing ResourceRef entries
• Updates goal to support --access flag for creating transactions with the new format
• Implements fallback to legacy foreign arrays when Access is not used
• Updates error messages to use "unavailable" instead of "invalid" for consistency

Reviewed Changes

Copilot reviewed 55 out of 56 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
util/fn.go	Adds functional programming utilities Map and MapErr for slice transformations
test/scripts/e2e_subs/tealprogs/xappreads.teal	Updates TEAL version and simplifies logic using assertions
test/scripts/e2e_subs/tealprogs/assets-escrow9.teal	New TEAL v9 program demonstrating access-aware asset operations
test/scripts/e2e_subs/shared-resources.py	Updates error message expectation
test/scripts/e2e_subs/e2e-app-x-app-reads.sh	Adds tests for --access flag functionality
test/scripts/e2e_subs/e2e-app-simulate.sh	Updates error message and box size for new quota
test/scripts/e2e_subs/app-assets.sh	Improves test reliability with proper error handling
test/scripts/e2e_subs/app-assets-access.sh	New comprehensive test suite for --access functionality
multiple test files	Updates API calls to use new RefBundle parameter structure
libgoal/transactions.go	Implements RefBundle type and access list attachment logic
libgoal/libgoal_test.go	Adds comprehensive tests for foreign reference and access resolution
data/transactions/logic/resources.go	Updates resource management for access lists and error messages
ledger/apply/application.go	Updates program checking to handle access lists