Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs/promises FileHandle does not exist #19165

Closed
lucacasonato opened this issue May 17, 2023 · 11 comments
Closed

fs/promises FileHandle does not exist #19165

lucacasonato opened this issue May 17, 2023 · 11 comments
Labels
bug Something isn't working correctly node API Related to various "node:*" modules APIs node compat

Comments

@lucacasonato
Copy link
Member

Deno:

> const fs = await import("node:fs/promises");
undefined
> const file3 = await fs.open("./README.md");
undefined
> file3
3
> file3
3
> file3.read
undefined

Node:

> const fs = await import("fs/promises");
undefined
>   const file3 = await fs.open("./tests/e2e_unit/testdata/file.txt");
undefined
> file3
FileHandle {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  close: [Function: close],
  [Symbol(kCapture)]: false,
  [Symbol(kHandle)]: FileHandle {},
  [Symbol(kFd)]: 24,
  [Symbol(kRefs)]: 1,
  [Symbol(kClosePromise)]: null
}
> file3.read
[Function: read]
@k-nasa
Copy link
Contributor

k-nasa commented May 20, 2023

I will work on this

@k-nasa
Copy link
Contributor

k-nasa commented May 20, 2023

Hello,

This is my first time contributing to Deno's OSS and I have a few questions.

Initially, I thought the issue could be resolved by simply adding something to the polyfill. However, I realized that FileHandle is not yet present. I understand that we need to add this and establish a sequence of steps for use with fs/promises and other.

However, I'm currently struggling to understand how Deno achieves Node compatibility. Could you please give me an overview of the architecture regarding this?

Also, I believe this document (https://github.com/denoland/deno/tree/main/ext/node/polyfills) is outdated and may not function adequately in its current state. I felt an update might be necessary concerning this.

I would like to propose working on fixing this issue. I would appreciate any advice or feedback on how best to proceed.

Thank you in advance.

@bartlomieju
Copy link
Member

Hey @k-nasa, thanks for interest in contributing. I will write up a longer rundown of things that need to happen to polyfill this API. I'll get back to you later tonight.

@k-nasa
Copy link
Contributor

k-nasa commented May 22, 2023

Thank you for your reply. looking forward to it. @bartlomieju

@bartlomieju
Copy link
Member

@k-nasa so I would suggest to start very small - focus on adding FileHandle class an implementing FileHandle.readFile call.

The actual FileHandle class in Node can be seen here:
https://github.com/nodejs/node/blob/959142a4652f7b6e90743be874fe9c065ed31d99/lib/internal/fs/promises.js#L132-L247

I suggest to skip the EventEmitterMixin(JSTransferable) part for now and only make it extend EventEmitter class. You can add it in ext/node/polyfills/internal/fs/handle.ts.

Then in ext/node/polyfills/_fs/_fs_readFile.ts in readFile function you will need to start with updating the checks for path variable to check if it is an instance of FileHandle.

Once you get that set up we can think about how to actually implement the API - right now I'm thinking that FileHandle should just call Deno.open internally and store the returned resource ID that we will later use for various operations.

Let me know if that's helpful or do you need more pointers to get started.

@k-nasa
Copy link
Contributor

k-nasa commented May 23, 2023

@bartlomieju Thank you! I think this is enough information to get started. I will try to work on it tonight.

From your explanation, it seems that "node compatibility" is not achieved by using Node internally, but by a TypeScript program that provides the same interface and behaves the same way as Node. Is the Rust language included in ext/node closely related to the Deno runtime?

@bartlomieju
Copy link
Member

From your explanation, it seems that "node compatibility" is not achieved by using Node internally, but by a TypeScript program that provides the same interface and behaves the same way as Node.

That's correct, we do not use Node.js internally, but instead provide a set of APIs that are available in Node.js but are implemented internally by Deno - the implementation is different, but the API surface is the same and we try to achieve the same behavior as in Node (though that's not always the case).

Is the Rust language included in ext/node closely related to the Deno runtime?

ext/node is using the same "op system" as the rest of Deno - if you take a look at ext/node/ops and runtime/ops you will see that the functions decorated with #[op] attribute look similar in both places. They do different things but they share a common API for calling from JavaScript to Rust.

@k-nasa
Copy link
Contributor

k-nasa commented May 25, 2023

I am working here
I'm working on a task list like this:

  • Change the return value of the open function to FileHandle
    • FileHandle
      • Add type
      • Implement methods
        • readFile
    • Change the return value of open
  • Change the return values of other functions that should return a FileHandle

@k-nasa
Copy link
Contributor

k-nasa commented May 25, 2023

@bartlomieju
I was wondering what is the reason for accepting FileHandle in readFile?
I thought it would be better to call ext/node/polyfills/_fs/_fs_readFile.ts from FileHandle when implementing the readFile method in FileHandle.

mmastrac added a commit that referenced this issue Jun 2, 2023
<!--
Before submitting a PR, please read https://deno.com/manual/contributing

1. Give the PR a descriptive title.

  Examples of good title:
    - fix(std/http): Fix race condition in server
    - docs(console): Update docstrings
    - feat(doc): Handle nested reexports

  Examples of bad title:
    - fix #7123
    - update docs
    - fix bugs

2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
   all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->


## WHY

ref: #19165

Node's fs/promises includes a FileHandle class, but deno does not. The
open function in Node's fs/promises returns a FileHandle, which provides
an IO interface to the file. However, deno's open function returns a
resource id.


### deno 

```js
> const fs = await import("node:fs/promises");
undefined
> const file3 = await fs.open("./README.md");
undefined
> file3
3
> file3.read
undefined
Node:
```

### Node
```js
> const fs = await import("fs/promises");
undefined
>   const file3 = await fs.open("./tests/e2e_unit/testdata/file.txt");
undefined
> file3
FileHandle {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  close: [Function: close],
  [Symbol(kCapture)]: false,
  [Symbol(kHandle)]: FileHandle {},
  [Symbol(kFd)]: 24,
  [Symbol(kRefs)]: 1,
  [Symbol(kClosePromise)]: null
}
> file3.read
[Function: read]
```


To be compatible with Node, deno's open function should also return a
FileHandle.

## WHAT

I have implemented the first step in adding a FileHandle.

- Changed the return value of the open function to a FileHandle object
- Implemented the readFile method in FileHandle
- Add test code


## What to do next
This PR is the first step in adding a FileHandle, and there are things
that should be done next.

- Add functionality equivalent to Node's FileHandle to FileHandle
(currently there is only readFile)

---------

Co-authored-by: Matt Mastracci <[email protected]>
@k-nasa
Copy link
Contributor

k-nasa commented Jun 3, 2023

FileHandle has this many methods
ref https://nodejs.org/api/fs.html#class-filehandle

FileHandle needs to implement these.

mmastrac added a commit that referenced this issue Jun 5, 2023
## WHY 

ref: #19165

The FileHandle class has many missing methods compared to node.
Add these.

## WHAT

- Add close method

---------

Co-authored-by: Matt Mastracci <[email protected]>
mmastrac pushed a commit that referenced this issue Jun 8, 2023
ref: #19165

The FileHandle class has many missing methods compared to node.
mmastrac pushed a commit that referenced this issue Jun 8, 2023
## WHY 

ref: #19165

The FileHandle class has many missing methods compared to node.

## WHAT


Add write method
bartlomieju pushed a commit that referenced this issue Jun 8, 2023
<!--
Before submitting a PR, please read https://deno.com/manual/contributing

1. Give the PR a descriptive title.

  Examples of good title:
    - fix(std/http): Fix race condition in server
    - docs(console): Update docstrings
    - feat(doc): Handle nested reexports

  Examples of bad title:
    - fix #7123
    - update docs
    - fix bugs

2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
   all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->


## WHY

ref: #19165

Node's fs/promises includes a FileHandle class, but deno does not. The
open function in Node's fs/promises returns a FileHandle, which provides
an IO interface to the file. However, deno's open function returns a
resource id.


### deno 

```js
> const fs = await import("node:fs/promises");
undefined
> const file3 = await fs.open("./README.md");
undefined
> file3
3
> file3.read
undefined
Node:
```

### Node
```js
> const fs = await import("fs/promises");
undefined
>   const file3 = await fs.open("./tests/e2e_unit/testdata/file.txt");
undefined
> file3
FileHandle {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  close: [Function: close],
  [Symbol(kCapture)]: false,
  [Symbol(kHandle)]: FileHandle {},
  [Symbol(kFd)]: 24,
  [Symbol(kRefs)]: 1,
  [Symbol(kClosePromise)]: null
}
> file3.read
[Function: read]
```


To be compatible with Node, deno's open function should also return a
FileHandle.

## WHAT

I have implemented the first step in adding a FileHandle.

- Changed the return value of the open function to a FileHandle object
- Implemented the readFile method in FileHandle
- Add test code


## What to do next
This PR is the first step in adding a FileHandle, and there are things
that should be done next.

- Add functionality equivalent to Node's FileHandle to FileHandle
(currently there is only readFile)

---------

Co-authored-by: Matt Mastracci <[email protected]>
bartlomieju pushed a commit that referenced this issue Jun 8, 2023
## WHY 

ref: #19165

The FileHandle class has many missing methods compared to node.
Add these.

## WHAT

- Add close method

---------

Co-authored-by: Matt Mastracci <[email protected]>
bartlomieju pushed a commit that referenced this issue Jun 8, 2023
ref: #19165

The FileHandle class has many missing methods compared to node.
bartlomieju pushed a commit that referenced this issue Jun 8, 2023
## WHY 

ref: #19165

The FileHandle class has many missing methods compared to node.

## WHAT


Add write method
@bartlomieju bartlomieju added the node API Related to various "node:*" modules APIs label Mar 4, 2024
@kt3k
Copy link
Member

kt3k commented Sep 10, 2024

Created #25554 for better visibility. Closing this one as the original issue has been resolved

@kt3k kt3k closed this as completed Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working correctly node API Related to various "node:*" modules APIs node compat
Projects
None yet
Development

No branches or pull requests

4 participants