Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 3 additions & 20 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -147,21 +147,6 @@ deps:

# artifacts

Comment thread
michaeldiamant marked this conversation as resolved.
# Regenerate algod swagger spec files
ALGOD_API_SWAGGER_SPEC := daemon/algod/api/swagger.json
ALGOD_API_FILES := $(shell find daemon/algod/api/server/common daemon/algod/api/server/v1 daemon/algod/api/spec/v1 -type f) \
daemon/algod/api/server/router.go
ALGOD_API_SWAGGER_INJECT := daemon/algod/api/server/lib/bundledSpecInject.go

# Note that swagger.json requires the go-swagger dep.
$(ALGOD_API_SWAGGER_SPEC): $(ALGOD_API_FILES) crypto/libs/$(OS_TYPE)/$(ARCH)/lib/libsodium.a
cd daemon/algod/api && \
PATH=$(GOPATH1)/bin:$$PATH \
go generate ./...

$(ALGOD_API_SWAGGER_INJECT): deps $(ALGOD_API_SWAGGER_SPEC) $(ALGOD_API_SWAGGER_SPEC).validated
Comment thread
michaeldiamant marked this conversation as resolved.
./daemon/algod/api/server/lib/bundle_swagger_json.sh

# Regenerate kmd swagger spec files
KMD_API_SWAGGER_SPEC := daemon/kmd/api/swagger.json
KMD_API_FILES := $(shell find daemon/kmd/api/ -type f | grep -v $(KMD_API_SWAGGER_SPEC))
Expand Down Expand Up @@ -191,15 +176,13 @@ $(KMD_API_SWAGGER_INJECT): deps $(KMD_API_SWAGGER_SPEC) $(KMD_API_SWAGGER_SPEC).

# generated files we should make sure we clean
GENERATED_FILES := \
$(ALGOD_API_SWAGGER_INJECT) \
$(KMD_API_SWAGGER_INJECT) \
$(ALGOD_API_SWAGGER_SPEC) $(ALGOD_API_SWAGGER_SPEC).validated \
$(KMD_API_SWAGGER_SPEC) $(KMD_API_SWAGGER_SPEC).validated

rebuild_swagger: deps
rebuild_kmd_swagger: deps
rm -f $(GENERATED_FILES)
# we need to invoke the make here since we want to ensure that the deletion and re-creating are sequential
make $(KMD_API_SWAGGER_INJECT) $(ALGOD_API_SWAGGER_INJECT)
make $(KMD_API_SWAGGER_INJECT)

# develop

Expand Down Expand Up @@ -327,7 +310,7 @@ dump: $(addprefix gen/,$(addsuffix /genesis.dump, $(NETWORKS)))
install: build
scripts/dev_install.sh -p $(GOPATH1)/bin

.PHONY: default fmt lint check_shell sanity cover prof deps build test fulltest shorttest clean cleango deploy node_exporter install %gen gen NONGO_BIN check-go-version rebuild_swagger
.PHONY: default fmt lint check_shell sanity cover prof deps build test fulltest shorttest clean cleango deploy node_exporter install %gen gen NONGO_BIN check-go-version rebuild_kmd_swagger

###### TARGETS FOR CICD PROCESS ######
include ./scripts/release/mule/Makefile.mule
Expand Down
66 changes: 0 additions & 66 deletions daemon/algod/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,69 +20,3 @@ Specifically, `uint64` types aren't strictly supported by OpenAPI. So we added a
## Why do we have algod.oas2.json and algod.oas3.yml?

We chose to maintain V2 and V3 versions of the spec because OpenAPI v3 doesn't seem to be widely supported. Some tools worked better with V3 and others with V2, so having both available has been useful. To reduce developer burdon, the v2 specfile is automatically converted v3 using [converter.swagger.io](http://converter.swagger.io/).

# Comments below are for v1 endpoints and are deprecated

## Components:

- `swagger.json` defines the API schema. However, server code in `api/v1/...`
currently serves as the ground truth, as the schema is generated from server code.
- to generate `swagger.json`, run `make build`. You may need to have `go-swagger`
installed. You can get it by running `make deps`.
- `api/client` is a package for internal (or external) libraries to interact with
the REST API. In particular, it should minimize dependencies.
- we currently use a non-swagger generated client. Why? The swagger generated client
pulls in too many dependencies (go-openapi, for instance) and unnecessary
functionality. Testing the swagger spec must be done another way. It seems that
unwrapped json raw types are sent on the wire (so not wrapped by responses),
so we don't need to decode them into responses.
- `api/v1/...` contains an implementation for the server. The swagger schema is auto-generated
(`cd api/; swagger generate spec -o ./swagger.json`) from server implementation code.
`api/v1/handlers` and `api/v1/models` should never be directly imported by external clients.
- or, run `go generate` in the `api` folder.


## Debugging/Engineering Notes:

- `go-swagger` does not generate `x-nullable` properties on model fields. We want them
so that we can generate models without pointers. (This is more compatible with the
current model we use. We may want to use pointers instead, eventually)
- make sure you populate the `default` property in order to generate a model
without a pointer field
- `go-swagger` does not support OpenAPI 3.0. It only supports OpenAPI 2.0. There
does not seem to be another tool that allows us to generate a swagger spec from
code. It may be worth writing our own, eventually.
- `go-swagger` does not support embedded structs.
- in fact, `go-swagger` is generally very strange. The source -> spec generation
looks fairly immature. Here are some (undocumented) tips:
- every `swagger:response` type must contain a single field (e.g. `Body` or
`Payload`) that is the actual data type you want to return. So the `response`
type is a wrapper, which makes sense, except the clients that `go-swagger`
generate automatically unwrap the underlying value. So this is very weird,
and undocumented.
- `swagger:route` is a less powerful version of the `swagger:operation`
annotation.
However, `swagger:operation` is much more finicky and not mature. When defining
the annotation, make sure it is precise yaml, and start the yaml section with
`---`. This means keeping track of tabs and whitespaces. This seems to be the
easiest way to define parameters without having to make explicit structs
(which we may want to do eventually anyways).
- don't deal with `go-swagger` codegen docs. Refer directly to
`https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#pathItemObject`/
- `go-swagger` does not support `regex` in path parameter path templating.
- complex parameter schemas are only supported in parameters `in:body`
- responses are distinct from definition objects (e.g. the former has a`description`
field, and headers). We always want to return a response in an operation. Returning
a model seems to work, but does not seem advised.
- `go-swagger` assumes `x-isnullable: true` and generates pointer files. If we ever
want to use a swagger generated client internally this may be a problem. Note that
`go-swagger` doesn't support a corresponding `x-isnullable` annotation. We can get around
that by using the `default` annotation and then find-and-replacing an `x-isnullable` into
the actual spec:
```//go:generate sed -i "" -e "s/\"default/\"x-nullable\": false, \"default/" ./swagger.json
//go:generate sed -i "" -e "s/object\",/object\", \"x-nullable\": false,/" ./swagger.json
```
- go-swagger does not seem to support simple string responses. They always get wrapped. (oh well)
e.g. [https://github.com/go-swagger/go-swagger/issues/1635]
- I've hardcoded a keylength into the spec for now, until I figure out how to tie that programatically
back into the server code (perhaps with a find-and-replace).
3 changes: 1 addition & 2 deletions daemon/algod/api/algod.oas2.json
Original file line number Diff line number Diff line change
Expand Up @@ -4197,8 +4197,7 @@
"description": "The minimum transaction fee (not per byte) required for the\ntxn to validate for the current network protocol.",
"type": "integer"
}
Comment thread
michaeldiamant marked this conversation as resolved.
},
"x-go-package": "github.com/algorand/go-algorand/daemon/algod/api/spec/v1"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what this was being used for in the v2 /params path spec, but it looked unnecessary so I deleted it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also looked and I'm unsure how x-go-package is used. In addition to general googling (e.g. https://swagger.io/specification/#specification-extensions), I reviewed these sources without finding a reference:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an additional sanity check, I confirmed Go SDK source code generation does not change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
}
},
"ApplicationResponse": {
Expand Down
6 changes: 2 additions & 4 deletions daemon/algod/api/algod.oas3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -802,8 +802,7 @@
"last-round",
"min-fee"
],
"type": "object",
"x-go-package": "github.com/algorand/go-algorand/daemon/algod/api/spec/v1"
"type": "object"
}
}
},
Expand Down Expand Up @@ -5404,8 +5403,7 @@
"last-round",
"min-fee"
],
"type": "object",
"x-go-package": "github.com/algorand/go-algorand/daemon/algod/api/spec/v1"
"type": "object"
}
}
},
Expand Down
2 changes: 1 addition & 1 deletion daemon/algod/api/server/common/handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ func VersionsHandler(ctx lib.ReqContext, context echo.Context) {
w.WriteHeader(http.StatusOK)
response := VersionsResponse{
Body: common.Version{
Versions: []string{"v1", "v2"},
Versions: []string{"v2"},
GenesisID: ctx.Node.GenesisID(),
GenesisHash: gh[:],
Build: common.BuildVersion{
Expand Down
15 changes: 0 additions & 15 deletions daemon/algod/api/server/lib/bundle_swagger_json.sh

This file was deleted.

Loading