CodeGen Discussion

[kelset]

Intro

With this issue I'd like to try and create a "one stop" for all the information available around the CodeGen: a tool to "automate" the compatibility between JS and native side. By using statically typed Javascript this tool can generate the interface files needed by Fabric and TurboModules - so that they know what will be send across the realms with confidence.
As @shergin said: "The biggest benefit of Codegen is not just type-safety, it's compile time type-safety. That means the code counterparts from both worlds can trust each other without any runtime checks, which means smaller code size and faster execution (less to load & less to execute)".

Terminology

• Fabric: just the UI layer re-architecture, to take full advantage of concurrent React architecture. (dedicated issue)
• TurboModules: re-architecture of NativeModules, also using JSI. (dedicated issue)
• JSI: JavaScript Interface, it's a unified lightweight general purpose API for (theoretically) any JavaScript virtual machine. It enables every other piece of the re-architecture. (dedicated issue)

TL;DR

From @axe-fb's blogpost, here's a temporary description of the CodeGen (please consider that this is not yet finalised, it may change in the future)

In both TurboModule and Fabric, interface available to JavaScript could be defined using Flow (or TypeScript). We can further leverage this interface definition to generate many of the C++ classes, and the interfaces/protocols for Java/ObjC implementations. For example, in case of TurboModules, the C++ class that wraps the Java/ObjC class and exposes the methods using a JSI object can be generated.
This will ensure that all JavaScript calls have implementations available on the native side, and will continue to ensure this with over the air updates like code push.

Available Materials

@TheSavior did an in-depth Twitch stream to explain the purpose of CodeGen.

@kelset wrote about it in part 1 of his blogpost series about the new architecture - and did a talk about the whole rearchitecture in April 2019 at React Edinburgh.

@kelset also did a more in-depth talk at React Advanced London in Oct 2019: youtube recording & slides.

Q&A

This is also a place for questions related to this effort and its direction.

[rhdeck]

I'd love to learn more about this initiative. I've developed a couple of codegen modules to accelerate native development in swift on iOS (react-native-swift-bridge) and Kotlin on Android (react-native-kotlin-bridge). Greasing these skids so native devs can make RN-enabled modules with a minimum of RN-focused fuss is my goal.

[rhdeck]

What is the directionality of the planned codegen?

For example: My work on RN-swift-bridge and RN-kotlin-bridge has been enabling native devs to write their work and minimize the "react native" overhead by generating the bridge and first-level JS bindings. So a swift developer writes swift, and then the ObjC and JS is auto-generated.

[kelset]

cc @TheSavior @rickhanlonii they may be able to help you

[rickhanlonii]

@rhdeck woah those projects are awesome!

The direction we're going is from javascript to native (so the source of truth is Flow/TS). Is there a particular reason you chose the other direction?

[rhdeck]

To my mind the most valuable native extension functionality - once you get past common core components - is about applying deep knowledge within the platform and bringing it to JS land for orchestration. JS developers know how to conduct, but they don't know how to play every instrument.

Enabling a specialist in a piece of native tech (say, CoreML) to bring that tech with a minimum of friction to the JS context seems like the direction of value, and is the reason I wrote the bridges the way I did.

In most cases the more expensive resources are the deeper specialists, so greasing their paths to (once) create modules reusable by the army of react devs who don't need to be versed in the more arcane aspects of the native interface seems like the direction of highest value.

All my opinion of course. Would love to understand the RN team's POV on the other direction and the codegen plan.

[elicwhite]

One of the main reasons our directionality was picked was to enforce cross platform compatibility. We want to push the ecosystem to provide modules that work across more platforms instead of only a single one. However, we need to continue supporting single platform modules. If we generated from native code, would we generate from java files? What about iOS only modules? If we generated from ObjC/Swift then what about java only modules? Do we write the codegen for each language so someone can use whatever they feel most comfortable in? Then other people in the ecosystem are less likely to be able to contribute to that module and the codegen will be fragmented across multiple implementations.

We could move the definition to cross platform c++ but we think people are even less likely to feel comfortable writing quality c++.

The common language for all React Native projects, on any platform not just including iOS/android but also VR, Windows, TV, Mac, Web is JavaScript. We figure that getting people to write a single interface file in JS would be the easiest choice to create modules that support all the needs. This is also a major part of why we are not using something like WebIDL as the source of truth.

[rhdeck]

That's a good manifesto! Will give the approach more thought.

[michalchudziak]

I like the idea. I think having such a CodeGen would lower the entry barrier for devs w/o native knowledge and encourage them to play with native modules a bit more, which seems like the right direction to follow.

[pzoltowski]

makes sense but on the other hand would be nice to have something similar to pybind11 that allows to annotate c++ headers and generate python bindings (call it e.g. jsbind11)

[yozman]

how to use this codegen? is there any operate sample?

[mrousavy]

Recently a lot of commits specifically for codegen landed on master, do we have some kind of progress update here? I believe android was a blocker a while ago, is codegen in a useable state on android now?

[karol-bisztyga]

Hey, I created a simple codegen-tool - maybe that will be helpful for someone.

I may describe what's going on there better soon, but the code is quite simple. I know that codegen is probably not yet out officially. There's a patch for it in my repo, which is a temporary solution for sure, but it worked for me in a real app. Also, I've not used generated code for JNI, but the generation itself works.

[michbil]

I'm trying to write NativeSpec for component of react-native-maps, and I have couple of questions about codegen.
I'm little bit frustrated by lack of documentation, so I am looking into react-native-codegen source code for answers.
I'm using typescript, so it's quite hard to find meaningful examples.

Link to my native spec gist

Conversion of props went mostly without problems, but i hit problem when trying to convert commands

export interface AIRMapNativeCommands {
 getCamera: (viewRef: React.ElementRef<any>) => Promise<Camera>;
  setCamera: (viewRef: React.ElementRef<any>, camera: Camera) => void;
  animateCamera: (
    viewRef: React.ElementRef<any>,
    camera: Camera,
    duration?: Float
  ) => void;
  ...
}
export const AIRMapCommands: AIRMapNativeCommands =
  codegenNativeCommands<AIRMapNativeCommands>({
    supportedCommands: [
      'getCamera',
      'setCamera',
      'animateCamera',
...
}

So the problem is, that for commands only primitive data types supported(int, strings, etc). Can not use any Object or Promise.

I'm getting something like this

 Error: Unsupported param type for method "setCamera", param "camera". Found Readonly

getCamera method is generated, but return type is incorrect

@protocol RCTAIRMapViewProtocol <NSObject>
- (void)getCamera;
@end

So far I browsed react-native-codegen's code, and found that commands support only BooleanTypeAnnotation, Int32TypeAnnotation, DoubleTypeAnnotation, FloatTypeAnnotation, StringTypeAnnotation. Any more complex data types are not supported. Is it limitation of the codegen? Or I am using it wrong?

[kelset]

@michbil the basic answer is that yes, only base types are supported, more info here: https://reactnative.dev/docs/0.68/new-architecture-library-intro#supported-flow-types

(same for typescript)

I would suggest you follow up from this in this: https://github.com/reactwg/react-native-new-architecture/discussions the Meta engineers want to use that discussion section for this type of conversation. (probably, in particular, here: https://github.com/reactwg/react-native-new-architecture/discussions/categories/deep-dive)

[michbil]

@kelset Thanks, I will start thread on discussion group, when I get write access here.

[kelset]

@michbil for TS specifically there's already this one btw: reactwg/react-native-new-architecture#27

[kelset]

hey folks, since we have an official deep dive discussions section open on this in the ReactWG New architecture, let's close this one and make the conversation progress over there: https://github.com/reactwg/react-native-new-architecture/discussions/categories/deep-dive (there are also some other sections, like Q&A, to ask more specific questions with an easier UX for replying)

Remember that you can apply to the WG (in case you are not in it yet) by applying to the form in the readme.