Skip to content

Commit

Permalink
feat: customize ipns dnsResolvers (#445)
Browse files Browse the repository at this point in the history
* feat(verified-fetch): customize ipns dnsResolvers

* chore: pr comment

Co-authored-by: Alex Potsides <[email protected]>

* test: fix dns-resolvers test

* chore: harden custom dns resolver test

* test: actually fix the custom dns resolver test

The test was passing when ran in isolation, but not in the group. dns caching was causing problems so i customized the url

* chore: firefox promise.any aggregate name is different

* fix: createVerifiedFetch passes through custom dnsResolvers

* docs: jsdoc fix from pr comment

Co-authored-by: Alex Potsides <[email protected]>

* test: move custom-dns-resolvers tests

* test: move content-type-parser test

* docs: typo

Co-authored-by: Alex Potsides <[email protected]>

* fix: dnsResolvers is passed in first arg to createVerifiedFetch

---------

Co-authored-by: Alex Potsides <[email protected]>
  • Loading branch information
SgtPooki and achingbrain authored Feb 26, 2024
1 parent e92086a commit 8f60822
Show file tree
Hide file tree
Showing 4 changed files with 108 additions and 6 deletions.
44 changes: 40 additions & 4 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,28 @@
* })
* ```
*
* ### Custom DNS resolvers
*
* If you don't want to leak DNS queries to the default resolvers, you can provide your own list of DNS resolvers to `createVerifiedFetch`.
*
* Note that you do not need to provide both a DNS-over-HTTPS and a DNS-over-JSON resolver, and you should prefer `dnsJsonOverHttps` resolvers for usage in the browser for a smaller bundle size. See https://github.com/ipfs/helia/tree/main/packages/ipns#example---using-dns-json-over-https for more information.
*
* @example Customizing DNS resolvers
*
* ```typescript
* import { createVerifiedFetch } from '@helia/verified-fetch'
* import { dnsJsonOverHttps, dnsOverHttps } from '@helia/ipns/dns-resolvers'
*
* const fetch = await createVerifiedFetch({
* gateways: ['https://trustless-gateway.link'],
* routers: ['http://delegated-ipfs.dev'],
* dnsResolvers: [
* dnsJsonOverHttps('https://my-dns-resolver.example.com/dns-json'),
* dnsOverHttps('https://my-dns-resolver.example.com/dns-query')
* ]
* })
* ```
*
* ### IPLD codec handling
*
* IPFS supports several data formats (typically referred to as codecs) which are included in the CID. `@helia/verified-fetch` attempts to abstract away some of the details for easier consumption.
Expand Down Expand Up @@ -472,7 +494,7 @@ import { createHeliaHTTP } from '@helia/http'
import { delegatedHTTPRouting } from '@helia/routers'
import { VerifiedFetch as VerifiedFetchClass } from './verified-fetch.js'
import type { Helia } from '@helia/interface'
import type { IPNSRoutingEvents, ResolveDnsLinkProgressEvents, ResolveProgressEvents } from '@helia/ipns'
import type { DNSResolver, IPNSRoutingEvents, ResolveDnsLinkProgressEvents, ResolveProgressEvents } from '@helia/ipns'
import type { GetEvents } from '@helia/unixfs'
import type { CID } from 'multiformats/cid'
import type { ProgressEvent, ProgressOptions } from 'progress-events'
Expand Down Expand Up @@ -508,6 +530,18 @@ export interface VerifiedFetch {
export interface CreateVerifiedFetchInit {
gateways: string[]
routers?: string[]

/**
* In order to parse DNSLink records, we need to resolve DNS queries. You can
* pass a list of DNS resolvers that we will provide to the @helia/ipns
* instance for you. You must construct them using the `dnsJsonOverHttps` or
* `dnsOverHttps` functions exported from `@helia/ipns/dns-resolvers`.
*
* We use cloudflare and google's dnsJsonOverHttps resolvers by default.
*
* @default [dnsJsonOverHttps('https://mozilla.cloudflare-dns.com/dns-query'),dnsJsonOverHttps('https://dns.google/resolve')]
*/
dnsResolvers?: DNSResolver[]
}

export interface CreateVerifiedFetchOptions {
Expand All @@ -516,6 +550,8 @@ export interface CreateVerifiedFetchOptions {
* provide will be passed the first set of bytes we receive from the network,
* and should return a string that will be used as the value for the
* `Content-Type` header in the response.
*
* @default undefined
*/
contentTypeParser?: ContentTypeParser
}
Expand Down Expand Up @@ -561,9 +597,9 @@ export interface VerifiedFetchInit extends RequestInit, ProgressOptions<BubbledP
* Create and return a Helia node
*/
export async function createVerifiedFetch (init?: Helia | CreateVerifiedFetchInit, options?: CreateVerifiedFetchOptions): Promise<VerifiedFetch> {
const contentTypeParser: ContentTypeParser | undefined = options?.contentTypeParser

let dnsResolvers: DNSResolver[] | undefined
if (!isHelia(init)) {
dnsResolvers = init?.dnsResolvers
init = await createHeliaHTTP({
blockBrokers: [
trustlessGateway({
Expand All @@ -574,7 +610,7 @@ export async function createVerifiedFetch (init?: Helia | CreateVerifiedFetchIni
})
}

const verifiedFetchInstance = new VerifiedFetchClass({ helia: init }, { contentTypeParser })
const verifiedFetchInstance = new VerifiedFetchClass({ helia: init }, { dnsResolvers, ...options })
async function verifiedFetch (resource: Resource, options?: VerifiedFetchInit): Promise<Response> {
return verifiedFetchInstance.fetch(resource, options)
}
Expand Down
5 changes: 3 additions & 2 deletions src/verified-fetch.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { car } from '@helia/car'
import { ipns as heliaIpns, type IPNS } from '@helia/ipns'
import { ipns as heliaIpns, type DNSResolver, type IPNS } from '@helia/ipns'
import { dnsJsonOverHttps } from '@helia/ipns/dns-resolvers'
import { unixfs as heliaUnixFs, type UnixFS as HeliaUnixFs, type UnixFSStats } from '@helia/unixfs'
import * as ipldDagCbor from '@ipld/dag-cbor'
Expand Down Expand Up @@ -43,6 +43,7 @@ interface VerifiedFetchComponents {
*/
interface VerifiedFetchInit {
contentTypeParser?: ContentTypeParser
dnsResolvers?: DNSResolver[]
}

interface FetchHandlerFunctionArg {
Expand Down Expand Up @@ -126,7 +127,7 @@ export class VerifiedFetch {
this.helia = helia
this.log = helia.logger.forComponent('helia:verified-fetch')
this.ipns = ipns ?? heliaIpns(helia, {
resolvers: [
resolvers: init?.dnsResolvers ?? [
dnsJsonOverHttps('https://mozilla.cloudflare-dns.com/dns-query'),
dnsJsonOverHttps('https://dns.google/resolve')
]
Expand Down
13 changes: 13 additions & 0 deletions test/content-type-parser.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import { stop } from '@libp2p/interface'
import { fileTypeFromBuffer } from '@sgtpooki/file-type'
import { expect } from 'aegir/chai'
import { filetypemime } from 'magic-bytes.js'
import Sinon from 'sinon'
import { fromString as uint8ArrayFromString } from 'uint8arrays/from-string'
import { createVerifiedFetch } from '../src/index.js'
import { VerifiedFetch } from '../src/verified-fetch.js'
import type { Helia } from '@helia/interface'
import type { CID } from 'multiformats/cid'
Expand All @@ -26,6 +28,17 @@ describe('content-type-parser', () => {
await stop(verifiedFetch)
})

it('is used when passed to createVerifiedFetch', async () => {
const contentTypeParser = Sinon.stub().resolves('text/plain')
const fetch = await createVerifiedFetch(helia, {
contentTypeParser
})
expect(fetch).to.be.ok()
const resp = await fetch(cid)
expect(resp.headers.get('content-type')).to.equal('text/plain')
await fetch.stop()
})

it('sets default content type if contentTypeParser is not passed', async () => {
verifiedFetch = new VerifiedFetch({
helia
Expand Down
52 changes: 52 additions & 0 deletions test/custom-dns-resolvers.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { stop } from '@libp2p/interface'
import { expect } from 'aegir/chai'
import Sinon from 'sinon'
import { createVerifiedFetch } from '../src/index.js'
import { VerifiedFetch } from '../src/verified-fetch.js'
import { createHelia } from './fixtures/create-offline-helia.js'
import type { Helia } from '@helia/interface'

describe('custom dns-resolvers', () => {
let helia: Helia

beforeEach(async () => {
helia = await createHelia()
})

afterEach(async () => {
await stop(helia)
})

it('is used when passed to createVerifiedFetch', async () => {
const customDnsResolver = Sinon.stub()

customDnsResolver.returns(Promise.resolve('/ipfs/QmVP2ip92jQuMDezVSzQBWDqWFbp9nyCHNQSiciRauPLDg'))

const fetch = await createVerifiedFetch({
gateways: ['http://127.0.0.1:8080'],
dnsResolvers: [customDnsResolver]
})
// error of walking the CID/dag because we haven't actually added the block to the blockstore
await expect(fetch('ipns://some-non-cached-domain.com')).to.eventually.be.rejected.with.property('errors')

expect(customDnsResolver.callCount).to.equal(1)
expect(customDnsResolver.getCall(0).args).to.deep.equal(['some-non-cached-domain.com', { onProgress: undefined }])
})

it('is used when passed to VerifiedFetch', async () => {
const customDnsResolver = Sinon.stub()

customDnsResolver.returns(Promise.resolve('/ipfs/QmVP2ip92jQuMDezVSzQBWDqWFbp9nyCHNQSiciRauPLDg'))

const verifiedFetch = new VerifiedFetch({
helia
}, {
dnsResolvers: [customDnsResolver]
})
// error of walking the CID/dag because we haven't actually added the block to the blockstore
await expect(verifiedFetch.fetch('ipns://some-non-cached-domain2.com')).to.eventually.be.rejected.with.property('errors').that.has.lengthOf(0)

expect(customDnsResolver.callCount).to.equal(1)
expect(customDnsResolver.getCall(0).args).to.deep.equal(['some-non-cached-domain2.com', { onProgress: undefined }])
})
})

0 comments on commit 8f60822

Please sign in to comment.