Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add user version of feature RPC #4781

Merged
merged 1 commit into from
Mar 5, 2024
Merged

Conversation

mvadari
Copy link
Collaborator

@mvadari mvadari commented Oct 23, 2023

High Level Overview of Change

This PR modifies the feature RPC to have a non-admin mode that allows conversion from the hex encodings of amendments to human-readable strings.

Example:

> {
    "command": "feature",
    "feature": "2E2FB9CF8A44EB80F4694D38AADAE9B8B7ADAFD2F092E10068E61C98C4F092B0"
  }
< {
    "result": {
      "2E2FB9CF8A44EB80F4694D38AADAE9B8B7ADAFD2F092E10068E61C98C4F092B0": {
        "enabled": false,
        "name": "fixUniversalNumber",
        "supported": true
      }
    },
    "type": "response"
  }

Context of Change

This has been requested by many explorer operators, so that the on-ledger hex amendment names can be decoded.

Type of Change

  • New feature (non-breaking change which adds functionality)

API Impact

  • Public API: New feature (new methods and/or new fields)

Test Plan

Added tests to handle this case.

@mvadari mvadari force-pushed the feature branch 3 times, most recently from 5857982 to 02f14b0 Compare October 23, 2023 17:09
@JST5000
Copy link
Contributor

JST5000 commented Oct 23, 2023

As an additional note, this is a feature I tried to find when I had when trying to set up our docker container for xrpl.js and xrpl-py CI since the Amendments stanza in the rippled.cfg file to force-enable amendments is really hard to read without the human-readable versions of amendments.

@dangell7
Copy link
Collaborator

This seems overkill.

import hashlib

def hash_string(k: str) -> str:
    return hashlib.sha512(k.encode('utf-8')).hexdigest().upper()[:64]
import * as crypto from 'crypto';

function sha512Hash(k: string): string {
  return crypto.createHash('sha512').update(k, 'utf8').digest('hex').toUpperCase().substring(0, 64);
}

@mvadari
Copy link
Collaborator Author

mvadari commented Oct 23, 2023

This seems overkill.

This works for going from the name to the hex, but not the other way around, since you can't reverse a hash. That's where this is needed, because that's what the ledger stores.

@pkcs8
Copy link

pkcs8 commented Oct 23, 2023

Would the non-admin version of this command have similar response shape as the admin version? I.e., if I skip the feature property, and just send {"command":"feature"}, it should return a map of all amendments known to this rippled sans voting data.

@mvadari
Copy link
Collaborator Author

mvadari commented Oct 23, 2023

Would the non-admin version of this command have similar response shape as the admin version? I.e., if I skip the feature property, and just send {"command":"feature"}, it should return a map of all amendments known to this rippled sans voting data.

I didn't add that here because it's a bit harder to do. But if that's a need, I can add it.

@pkcs8
Copy link

pkcs8 commented Oct 23, 2023

I read rippled websocket url from the environment, and often switch and use admin and non-admin ports interchangeably. I'm not sure now complicated it would be, but would be great if response shape remains same so I don't have to write conditions. I like how account_objects command works with its type filter.

@shortthefomo
Copy link

I didn't add that here because it's a bit

I'm actually a +1 for this I need a way to figure the list of amendments on the network. Currently using external API for mainnet or hardcoded. Would help a lot.

@mvadari
Copy link
Collaborator Author

mvadari commented Oct 24, 2023

@pkcs8 @lathanbritz I have switched to the existing admin feature format (removing the admin fields of course) and added the ability to get a list of all features in 43a1ce7.

@mvadari
Copy link
Collaborator Author

mvadari commented Oct 24, 2023

@intelliot can I add @ximinez and @thejohnfreeman as reviewers (if they're okay with that)?

@intelliot
Copy link
Collaborator

@mvadari I requested ximinez and thejohnfreeman but the PRs listed here are higher priority: https://github.com/XRPLF/rippled/milestone/8 <- these are 2.0 release blockers

This PR (#4781) is currently planned for 2.1.0

@intelliot intelliot added this to the 2024 release milestone Oct 24, 2023
@dangell7
Copy link
Collaborator

We don't like this on feature endpoint. We believe opening up feature like this is dangerous. We have opted to move this to the ServerDefinitions.cpp file.

What are your thoughts on this?

I know this goes against what XRPScan asked for but in our opinion its a "safer" solution.

@mvadari
Copy link
Collaborator Author

mvadari commented Oct 31, 2023

We don't like this on feature endpoint. We believe opening up feature like this is dangerous. We have opted to move this to the ServerDefinitions.cpp file.

What are your thoughts on this?

I know this goes against what XRPScan asked for but in our opinion its a "safer" solution.

This code change is pretty small and therefore easy to audit and test from a security perspective. There is also prior art for functions that have separate user versions and admin versions, like server_info, and there have been no issues with those. I'm personally not concerned, but there's also a reason we go through code review 🙂

src/ripple/app/misc/impl/AmendmentTable.cpp Show resolved Hide resolved
src/ripple/rpc/handlers/Feature1.cpp Outdated Show resolved Hide resolved
src/ripple/rpc/handlers/Feature1.cpp Outdated Show resolved Hide resolved
src/ripple/rpc/handlers/Feature1.cpp Outdated Show resolved Hide resolved
@mvadari mvadari requested a review from ximinez November 29, 2023 17:55
Copy link
Collaborator

@ximinez ximinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and useful

@intelliot intelliot added API Change Feature Request Used to indicate requests to add new features labels Dec 9, 2023
@ximinez
Copy link
Collaborator

ximinez commented Feb 21, 2024

@mvadari Is this ready to merge?

@mvadari
Copy link
Collaborator Author

mvadari commented Feb 21, 2024

@mvadari Is this ready to merge?

@ximinez with Bronek's ✅ , yep - I think it was just waiting on the 2.1 release.

@ximinez ximinez added the Passed Passed code review & PR owner thinks it's ready to merge. Perf sign-off may still be required. label Feb 21, 2024
@ximinez
Copy link
Collaborator

ximinez commented Feb 21, 2024

[see following comment]

@mvadari
Copy link
Collaborator Author

mvadari commented Feb 21, 2024

Proposed commit message:

feat: add user version of `feature` RPC (#4781)

* hides potentially sensitive data
* uses same formatting as admin RPC

@ximinez nit: I'd switch the order of the bullets

feat: add user version of `feature` RPC (#4781):

* uses same formatting as admin RPC
* hides potentially sensitive data

@intelliot
Copy link
Collaborator

From my perspective, @ximinez or @seelabs are welcome to merge this. (no big rush though)

mvadari added a commit to mvadari/rippled that referenced this pull request Feb 29, 2024
* uses same formatting as admin RPC
* hides potentially sensitive data
mvadari added a commit to mvadari/rippled that referenced this pull request Feb 29, 2024
* uses same formatting as admin RPC
* hides potentially sensitive data
mvadari added a commit to mvadari/rippled that referenced this pull request Mar 1, 2024
* uses same formatting as admin RPC
* hides potentially sensitive data
* uses same formatting as admin RPC
* hides potentially sensitive data
@ximinez ximinez merged commit 22b7518 into XRPLF:develop Mar 5, 2024
17 checks passed
@mvadari mvadari deleted the feature branch March 5, 2024 21:46
@ximinez ximinez mentioned this pull request Mar 6, 2024
1 task
@ximinez ximinez mentioned this pull request Apr 4, 2024
1 task
legleux added a commit to legleux/rippled that referenced this pull request Apr 12, 2024
* Price Oracle (XLS-47d): (XRPLF#4789) (XRPLF#4789)

Implement native support for Price Oracles.

 A Price Oracle is used to bring real-world data, such as market prices,
 onto the blockchain, enabling dApps to access and utilize information
 that resides outside the blockchain.

 Add Price Oracle functionality:
 - OracleSet: create or update the Oracle object
 - OracleDelete: delete the Oracle object

 To support this functionality add:
 - New RPC method, `get_aggregate_price`, to calculate aggregate price for a token pair of the specified oracles
 - `ltOracle` object

 The `ltOracle` object maintains:
 - Oracle Owner's account
 - Oracle's metadata
 - Up to ten token pairs with the scaled price
 - The last update time the token pairs were updated

 Add Oracle unit-tests

* fix compile error on gcc 13: (XRPLF#4932)

The compilation fails due to an issue in the initializer list
of an optional argument, which holds a vector of pairs.
The code compiles correctly on earlier gcc versions, but fails on gcc 13.

* Set version to 2.2.0-b1

* Remove default ctors from SecretKey and PublicKey: (XRPLF#4607)

* It is now an invariant that all constructed Public Keys are valid,
  non-empty and contain 33 bytes of data.
* Additionally, the memory footprint of the PublicKey class is reduced.
  The size_ data member is declared as static.
* Distinguish and identify the PublisherList retrieved from the local
  config file, versus the ones obtained from other validators.
* Fixes XRPLF#2942

* Fast base58 codec: (XRPLF#4327)

This algorithm is about an order of magnitude faster than the existing
algorithm (about 10x faster for encoding and about 15x faster for
decoding - including the double hash for the checksum). The algorithms
use gcc's int128 (fast MS version will have to wait, in the meantime MS
falls back to the slow code).

* feat: add user version of `feature` RPC (XRPLF#4781)

* uses same formatting as admin RPC
* hides potentially sensitive data

* build: add STCurrency.h to xrpl_core to fix clio build (XRPLF#4939)

* Embed patched recipe for RocksDB 6.29.5 (XRPLF#4947)

* fix: order book update variable swap: (XRPLF#4890)

This is likely the result of a typo when the code was simplified.

* Fix workflows (XRPLF#4948)

The problem was `CONAN_USERNAME` environment variable, which Conan 1.x uses as the default user in package references.

* Upgrade to xxhash 0.8.2 as a Conan requirement, enable SIMD hashing (XRPLF#4893)

We are currently using old version 0.6.2 of `xxhash`, as a verbatim copy and paste of its header file `xxhash.h`. Switch to the more recent version 0.8.2. Since this version is in Conan Center (and properly protects its ABI by keeping the state object incomplete), add it as a Conan requirement. Switch to the SIMD instructions (in the new `XXH3` family) supported by the new version.

* Update remaining actions (XRPLF#4949)

Downgrade {upload,download}-artifact action to v3 because of unreliability with v4.

* Install more public headers (XRPLF#4940)

Fixes some mistakes in XRPLF#4885

* test: Env unit test RPC errors return a unique result: (XRPLF#4877)

* telENV_RPC_FAILED is a new code, reserved exclusively
  for unit tests when RPC fails. This will
  make those types of errors distinct and easier to test
  for when expected and/or diagnose when not.
* Output RPC command result when result is not expected.

* Fix workflows (XRPLF#4951)

- Update container for Doxygen workflow. Matches Linux workflow, with newer GLIBC version required by newer actions.
- Fixes macOS workflow to install and configure Conan correctly. Still fails on tests, but that does not seem attributable to the workflow.

* perf: improve `account_tx` SQL query: (XRPLF#4955)

The witness server makes heavily use of the `account_tx` RPC command. Perf
testing showed that the SQL query used by `account_tx` became unacceptably slow
when the DB was large and there was a `marker` parameter. The plan for the query
showed only indexed reads. This appears to be an issue with the internal SQLite
optimizer. This patch rewrote the query to use `UNION` instead of `OR` and
significantly improves performance. See RXI-896 and RIPD-1847 for more details.

* `fixEmptyDID`: fix amendment to handle empty DID edge case: (XRPLF#4950)

This amendment fixes an edge case where an empty DID object can be
created. It adds an additional check to ensure that DIDs are
non-empty when created, and returns a `tecEMPTY_DID` error if the DID
would be empty.

* Enforce no duplicate slots from incoming connections: (XRPLF#4944)

We do not currently enforce that incoming peer connection does not have
remote_endpoint which is already used (either by incoming or outgoing
connection), hence already stored in slots_. If we happen to receive a
connection from such a duplicate remote_endpoint, it will eventually result in a
crash (when disconnecting) or weird behavior (when updating slot state), as a
result of an apparently matching remote_endpoint in slots_ being used by a
different connection.

* Remove zaphod.alloy.ee hub from default server list: (XRPLF#4903)

Remove the zaphod.alloy.ee hubs from the bootstrap and default configuration after 5 years. It has been an honor to run these servers, but it is now time for another entity to step into this role.

The zaphod servers will be taken offline in a phased manner keeping all those who have peering arrangements informed.

These would be the preferred attributes of a boostrap set of hubs:

    1. Commitment to run the hubs for a minimum of 2 years
    2. Highly available
    3. Geographically dispersed
    4. Secure and up to date
    5. Committed to ensure that peering information is kept private

* Write improved `forAllApiVersions` used in NetworkOPs (XRPLF#4833)

* Don't reach consensus as quickly if no other proposals seen: (XRPLF#4763)

This fixes a case where a peer can desync under a certain timing
circumstance--if it reaches a certain point in consensus before it receives
proposals. 

This was noticed under high transaction volumes. Namely, when we arrive at the
point of deciding whether consensus is reached after minimum establish phase
duration but before having received any proposals. This could be caused by
finishing the previous round slightly faster and/or having some delay in
receiving proposals. Existing behavior arrives at consensus immediately after
the minimum establish duration with no proposals. This causes us to desync
because we then close a non-validated ledger. The change in this PR causes us to
wait for a configured threshold before making the decision to arrive at
consensus with no proposals. This allows validators to catch up and for brief
delays in receiving proposals to be absorbed. There should be no drawback since,
with no proposals coming in, we needn't be in a huge rush to jump ahead.

* fixXChainRewardRounding: round reward shares down: (XRPLF#4933)

When calculating reward shares, the amount should always be rounded
down. If the `fixUniversalNumber` amendment is not active, this works
correctly. If it is not active, then the amount is incorrectly rounded
up. This patch introduces an amendment so it will be rounded down.

* Remove unused files

* Remove packaging scripts

* Consolidate external libraries

* Simplify protobuf generation

* Rename .hpp to .h

* Format formerly .hpp files

* Rewrite includes

$ find src/ripple/ src/test/ -type f -exec sed -i 's:include\s*["<]ripple/\(.*\)\.h\(pp\)\?[">]:include <ripple/\1.h>:' {} +

* Fix source lists

* Add markers around source lists

* fix: improper handling of large synthetic AMM offers:

A large synthetic offer was not handled correctly in the payment engine.
This patch fixes that issue and introduces a new invariant check while
processing synthetic offers.

* Set version to 2.1.1

* chore: change Github Action triggers for build/test jobs (XRPLF#4956)

Github Actions for the build/test jobs (nix.yml, mac.yml, windows.yml) will only run on branches that build packages (develop, release, master), and branches with names starting with "ci/". This is intended as a compromise between disabling CI jobs on personal forks entirely, and having the jobs run as a free-for-all. Note that it will not affect PR jobs at all.

* Address compiler warnings

* Fix search for protoc

* chore: Default validator-keys-tool to master branch: (XRPLF#4943)

* master is the default branch for that project. There's no point in
  using develop.

* Remove unused lambdas from MultiApiJson_test

* fix Conan component reference typo

* Set version to 2.2.0-b2

* bump version

* 2.2.3

* 2.2.4

* 2.2.5

---------

Co-authored-by: Gregory Tsipenyuk <[email protected]>
Co-authored-by: seelabs <[email protected]>
Co-authored-by: Chenna Keshava B S <[email protected]>
Co-authored-by: Mayukha Vadari <[email protected]>
Co-authored-by: John Freeman <[email protected]>
Co-authored-by: Bronek Kozicki <[email protected]>
Co-authored-by: Ed Hennis <[email protected]>
Co-authored-by: Olek <[email protected]>
Co-authored-by: Alloy Networks <[email protected]>
Co-authored-by: Mark Travis <[email protected]>
Co-authored-by: Gregory Tsipenyuk <[email protected]>
@seelabs seelabs mentioned this pull request Jun 4, 2024
1 task
sophiax851 pushed a commit to sophiax851/rippled that referenced this pull request Jun 12, 2024
* uses same formatting as admin RPC
* hides potentially sensitive data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Change Clio Reviewed Feature Request Used to indicate requests to add new features Passed Passed code review & PR owner thinks it's ready to merge. Perf sign-off may still be required. Perf Test Desired (Optional) RippleX Perf Team should look at this PR. The PR will not necessarily wait for testing to finish
Projects
Status: 🚢 Released in 2.2.0
Development

Successfully merging this pull request may close these issues.