Skip to content

add handling for non-utf 8 asset names #2555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tsachiherman merged 6 commits into from
Jul 16, 2021
Merged

add handling for non-utf 8 asset names #2555

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2141fc0
Move assets names/units/url to have a b64 version.
tsachiherman Jul 14, 2021
7055b9b
add goal support
tsachiherman Jul 14, 2021
292c864
error handling
tsachiherman Jul 14, 2021
21c91b4
update asset test version
tsachiherman Jul 14, 2021
aa9ee96
Update per review
tsachiherman Jul 15, 2021
94745c0
Few more updates per reviewer's feedback.
tsachiherman Jul 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions cmd/goal/account.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -591,22 +591,28 @@ func printAccountInfo(client libgoal.Client, address string, account v1.Account)
}
for _, id := range heldAssets {
assetHolding := account.Assets[id]
assetParams, err := client.AssetInformation(id)
assetParams, err := client.AssetInformationV2(id)
if err != nil {
hasError = true
fmt.Fprintf(errorReport, "Error: Unable to retrieve asset information for asset %d referred to by account %s: %v\n", id, address, err)
fmt.Fprintf(report, "\tID %d, error\n", id)
}

amount := assetDecimalsFmt(assetHolding.Amount, assetParams.Decimals)
amount := assetDecimalsFmt(assetHolding.Amount, uint32(assetParams.Params.Decimals))

assetName := assetParams.AssetName
var assetName string
if assetParams.Params.Name != nil {
assetName = *assetParams.Params.Name
}
if len(assetName) == 0 {
assetName = "<unnamed>"
}
_, assetName = unicodePrintable(assetName)

unitName := assetParams.UnitName
var unitName string
if assetParams.Params.UnitName != nil {
unitName = *assetParams.Params.UnitName
}
if len(unitName) == 0 {
unitName = "units"
}
Expand Down
21 changes: 18 additions & 3 deletions daemon/algod/api/algod.oas2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1416,9 +1416,14 @@
"format": "byte"
},
"name": {
"description": "\\[an\\] Name of this asset, as supplied by the creator.",
"description": "\\[an\\] Name of this asset, as supplied by the creator. Included only when the asset name is composed of printable utf-8 characters.",
"type": "string"
},
"name_b64": {
"description": "\\[an64\\] base64 encoded name of this asset, as supplied by the creator.",
"type": "string",
"x-algorand-format": "base64"
},
"reserve": {
"description": "\\[r\\] Address of account holding reserve (non-minted) units of this asset.",
"type": "string"
Expand All @@ -1429,12 +1434,22 @@
"x-algorand-format": "uint64"
},
"unit-name": {
"description": "\\[un\\] Name of a unit of this asset, as supplied by the creator.",
"description": "\\[un\\] Name of a unit of this asset, as supplied by the creator. Included only when the name of a unit of this asset is composed of printable utf-8 characters.",
"type": "string"
},
"unit-name_b64": {
"description": "\\[un64\\] base64 encoded name of a unit of this asset, as supplied by the creator.",
"type": "string",
"x-algorand-format": "base64"
},
"url": {
"description": "\\[au\\] URL where more information about the asset can be retrieved.",
"description": "\\[au\\] URL where more information about the asset can be retrieved. Included only when the URL is composed of printable utf-8 characters.",
"type": "string"
},
"url_b64": {
"description": "\\[au64\\] base64 encoded URL where more information about the asset can be retrieved.",
"type": "string",
"x-algorand-format": "base64"
}
}
},
Expand Down
21 changes: 18 additions & 3 deletions daemon/algod/api/algod.oas3.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1027,9 +1027,14 @@
"type": "string"
},
"name": {
"description": "\\[an\\] Name of this asset, as supplied by the creator.",
"description": "\\[an\\] Name of this asset, as supplied by the creator. Included only when the asset name is composed of printable utf-8 characters.",
"type": "string"
},
"name_b64": {
"description": "\\[an64\\] base64 encoded name of this asset, as supplied by the creator.",
"type": "string",
"x-algorand-format": "base64"
},
"reserve": {
"description": "\\[r\\] Address of account holding reserve (non-minted) units of this asset.",
"type": "string"
Expand All @@ -1040,12 +1045,22 @@
"x-algorand-format": "uint64"
},
"unit-name": {
"description": "\\[un\\] Name of a unit of this asset, as supplied by the creator.",
"description": "\\[un\\] Name of a unit of this asset, as supplied by the creator. Included only when the name of a unit of this asset is composed of printable utf-8 characters.",
"type": "string"
},
"unit-name_b64": {
"description": "\\[un64\\] base64 encoded name of a unit of this asset, as supplied by the creator.",
"type": "string",
"x-algorand-format": "base64"
},
"url": {
"description": "\\[au\\] URL where more information about the asset can be retrieved.",
"description": "\\[au\\] URL where more information about the asset can be retrieved. Included only when the URL is composed of printable utf-8 characters.",
"type": "string"
},
"url_b64": {
"description": "\\[au64\\] base64 encoded URL where more information about the asset can be retrieved.",
"type": "string",
"x-algorand-format": "base64"
}
},
"required": [
Expand Down
23 changes: 19 additions & 4 deletions daemon/algod/api/server/v1/handlers/handlers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ import (
"io"
"net/http"
"strconv"
"strings"
"time"
"unicode"
"unicode/utf8"

"github.com/labstack/echo/v4"

Expand Down Expand Up @@ -163,16 +164,30 @@ func participationKeysEncode(r basics.AccountData) *v1.Participation {
return &apiParticipation
}

// printableUTF8OrEmpty checks to see if the entire string is a UTF8 printable string.
// If this is the case, the string is returned as is. Otherwise, the empty string is returned.
func printableUTF8OrEmpty(in string) string {
// iterate throughout all the characters in the string to see if they are all printable.
// when range iterating on go strings, go decode each element as a utf8 rune.
for _, c := range in {
// is this a printable character, or invalid rune ?
if c == utf8.RuneError || !unicode.IsPrint(c) {
return ""
}
}
return in
}

func modelAssetParams(creator basics.Address, params basics.AssetParams) v1.AssetParams {
paramsModel := v1.AssetParams{
Total: params.Total,
DefaultFrozen: params.DefaultFrozen,
Decimals: params.Decimals,
}

paramsModel.UnitName = strings.TrimRight(string(params.UnitName[:]), "\x00")
paramsModel.AssetName = strings.TrimRight(string(params.AssetName[:]), "\x00")
paramsModel.URL = strings.TrimRight(string(params.URL[:]), "\x00")
paramsModel.UnitName = printableUTF8OrEmpty(params.UnitName)
paramsModel.AssetName = printableUTF8OrEmpty(params.AssetName)
paramsModel.URL = printableUTF8OrEmpty(params.URL)
if params.MetadataHash != [32]byte{} {
paramsModel.MetadataHash = params.MetadataHash[:]
}
Expand Down
9 changes: 6 additions & 3 deletions daemon/algod/api/server/v2/account.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -430,9 +430,12 @@ func AssetParamsToAsset(creator string, idx basics.AssetIndex, params *basics.As
Total: params.Total,
Decimals: uint64(params.Decimals),
DefaultFrozen: &frozen,
Name: strOrNil(params.AssetName),
UnitName: strOrNil(params.UnitName),
Url: strOrNil(params.URL),
Name: strOrNil(printableUTF8OrEmpty(params.AssetName)),
NameB64: strOrNil(base64.StdEncoding.EncodeToString([]byte(params.AssetName))),
Copy link
Copy Markdown
Contributor

@brianolson brianolson Jul 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we were only going to return one copy. Plain if valid, b64 otherwise.

Copy link
Copy Markdown
Contributor Author

@tsachiherman tsachiherman Jul 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we're storing one copy, but returning (sometimes) both, so that clients that use b64 could always decode that and could ignore the non-b64 version.

UnitName: strOrNil(printableUTF8OrEmpty(params.UnitName)),
UnitNameB64: strOrNil(base64.StdEncoding.EncodeToString([]byte(params.UnitName))),
Url: strOrNil(printableUTF8OrEmpty(params.URL)),
UrlB64: strOrNil(base64.StdEncoding.EncodeToString([]byte(params.URL))),
Clawback: addrOrNil(params.Clawback),
Freeze: addrOrNil(params.Freeze),
Manager: addrOrNil(params.Manager),
Expand Down
Loading