-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: run create-t3-app in existing projects to add missing parts #116
Comments
As cool as this sounds, it would be nye impossible to do. IMO this would be a different tool (and should probably be built by the maintainers of the specific deps) |
I love this idea, but how would the implementation work? There is the possibility to break existing code etc. |
Yeah with the current architecture this is not possible since we copy entire files over. If you have clashing files all your current progress would be overwritten. |
Trying to think through the steps required here:
That's going to be a little difficult. And by a little, I mean damn near impossible. If anyone can find a way to make this work, I will be absurdly impressed. |
First of all I want to say thank you all for the great work and effort you guys put into this. I'm loving this stack so far and I see great potential, I'm definitely using it when I need to do some new project. The topic of this issue is quite complex stuff and as you said, nearly impossible because once the files go to user land this tool will lose control and won't be able to easily modify those files. With this clarified, I have some ideas to throw into the mix. Sorry if I don't explain something properly as English is not my native language. Imagine that each addon can define 2 different types of template files:
The addon would perform operations upon installing, which could be copying base files (currently done) and applying patches to certain files (those could be conditionally applied based on installed packages aswell). The installation process could have some sort of flags in order to know whether it's a fresh/initial installation or a later installation of a single addon (though this installation process should always be aware of the already installed addons). Difficulties:
Extra benefits:
Another semi-related idea:
I'll keep thinking on this. |
To expand a bit the idea with patches, take the // src/server/router/context.ts
import * as trpc from "@trpc/server";
import * as trpcNext from "@trpc/server/adapters/next";
export const createContext = async (opts?: trpcNext.CreateNextContextOptions) => {
const req = opts?.req;
const res = opts?.res;
return {
req,
res,
};
};
type Context = trpc.inferAsyncReturnType<typeof createContext>;
export const createRouter = () => trpc.router<Context>(); Then we could have 2 patches:
diff --git a/template/addons/trpc/base-context.ts b/template/addons/trpc/base-context.ts
index f66d8d9..b6cbb39 100644
--- a/template/addons/trpc/base-context.ts
+++ b/template/addons/trpc/base-context.ts
@@ -2,6 +2,8 @@
import * as trpc from "@trpc/server";
import * as trpcNext from "@trpc/server/adapters/next";
+import { prisma } from "../db/client";
+
export const createContext = async (opts?: trpcNext.CreateNextContextOptions) => {
const req = opts?.req;
const res = opts?.res;
@@ -9,6 +11,7 @@ export const createContext = async (opts?: trpcNext.CreateNextContextOptions) =>
return {
req,
res,
+ prisma,
};
};
diff --git a/template/addons/trpc/base-context.ts b/template/addons/trpc/base-context.ts
index f66d8d9..36fe3ab 100644
--- a/template/addons/trpc/base-context.ts
+++ b/template/addons/trpc/base-context.ts
@@ -1,14 +1,21 @@
// src/server/router/context.ts
import * as trpc from "@trpc/server";
import * as trpcNext from "@trpc/server/adapters/next";
+import { unstable_getServerSession as getServerSession } from "next-auth";
+
+import { authOptions as nextAuthOptions } from "../../pages/api/auth/[...nextauth]";
export const createContext = async (opts?: trpcNext.CreateNextContextOptions) => {
const req = opts?.req;
const res = opts?.res;
+ const session =
+ req && res && (await getServerSession(req, res, nextAuthOptions));
+
return {
req,
res,
+ session,
};
};
Applying them one after the other: ➜ git apply --3way template/addons/trpc/context-prisma.diff
Applied patch to 'template/addons/trpc/base-context.ts' cleanly.
➜ git apply --3way template/addons/trpc/context-auth.diff
Applied patch to 'template/addons/trpc/base-context.ts' with conflicts.
U template/addons/trpc/base-context.ts And would yield the following file with some conflicts: // src/server/router/context.ts
import * as trpc from "@trpc/server";
import * as trpcNext from "@trpc/server/adapters/next";
import { unstable_getServerSession as getServerSession } from "next-auth";
import { authOptions as nextAuthOptions } from "../../pages/api/auth/[...nextauth]";
import { prisma } from "../db/client";
export const createContext = async (opts?: trpcNext.CreateNextContextOptions) => {
const req = opts?.req;
const res = opts?.res;
const session =
req && res && (await getServerSession(req, res, nextAuthOptions));
return {
req,
res,
<<<<<<< ours
prisma,
=======
session,
>>>>>>> theirs
};
};
type Context = trpc.inferAsyncReturnType<typeof createContext>;
export const createRouter = () => trpc.router<Context>(); This possible conflicts would be a problem, not sure how they could be handled during installation. |
Sorry to spam, but I have dreamed about this. Let's asume for a moment that the files are generated using a template engine such as handlebars, ejs, eta... Then the same template would be used to generate the same file with different configs. Asume aswell that in the template variables we can find which addons are being installed so we can add code conditionally. With this in mind, take that the user does a fresh with just one addon (say tRPC). The templates would yield the files without any code for any other addon. If then the user wants to install a new addon (say Prisma), the installer code would do the following for each installed addon (just tTRPC in this case):
Considerations:
For the examples I'm going to use EJS and some simpler code as it was a file in the tRPC addon. Take that we have the following template file in import * as trpc from "@trpc/server";
<% if (addons.nextAuth.inUse) { %>
import { authOptions as nextAuthOptions } from "../../pages/api/auth/[...nextauth]";
<% } %>
<% if (addons.prisma.inUse) { %>
import { prisma } from "../db/client";
<% } %> If the user initially installs just tRPC, this would yield: import * as trpc from "@trpc/server"; If the user then installs the Prisma addon, following the above described flow we would get:
import * as trpc from "@trpc/server";
import * as trpc from "@trpc/server";
+import { prisma } from "../db/client";
diff --git a/template/addons/trpc/context.ts b/template/addons/trpc/context.ts
index 8e7351d..81efa94 100644
--- a/template/addons/trpc/context.ts
+++ b/template/addons/trpc/context.ts
@@ -1 +1,2 @@
-import * as trpc from "@trpc/server";
\ No newline at end of file
+import * as trpc from "@trpc/server";
+import { prisma } from "../db/client";
Then for step 4 we could encounter 2 cases: (a). If the user has not modified the file, applying the patch yields no conflicts and we would get the proper merged file: import * as trpc from "@trpc/server";
import { prisma } from "../db/client"; (b). If the user has modified the file, let's say to this: import * as trpc from "@trpc/server";
import * as userLandPackage from "@userland/package"; Then applying the patch would result in conflicts, that would need to be managed by the user: import * as trpc from "@trpc/server";
<<<<<<< ours
import * as userLandPackage from "@userland/package";
=======
import { prisma } from "../db/client";
>>>>>>> theirs I hope the above makes sense 🤔 |
@ochicf I think if we choose to let the user to handle any of the merge conflict by using the patch, it will break the purpose of this project itself. Since the purpose of it is to simplify the process of creating the boilerplate so people can use it and add what they need after that. and by letting the user resolve the merge conflict manually will make it much painful to the user itself since probably they will have a ton of merge conflict. |
@krsbx you have a really good point. Maybe I wasn't clear enough, but with my last comment I was disregarding my own idea of patching files when creating the project the first time. Hence, creating the initial boilerplate would result in a conflict free project, as we would be just using a template engine to handle conditionals and other injected stuff in the generated files. But to solve the problem of incrementally add and modify the files once the project is created, I cannot think of anything that could ensure that we don't potentially get conflicts because of the freedom by the user to modify the files freely. My proposal only would yield conflicts when installing addons afterwards, and if the user has modified some of files affected by the new install... which IMHO it's still better than have to do all modifications manually. Of course It's far from ideal and would add other problems and complexities, I was just trying to throw ideas into the mix to see if we find a way to handle this and it's worth implementing it (which I have some doubts ATM). |
So you really got me thinking about how this could be possible. My curiosity has gotten the better of me and I am experimenting with using |
I've been playing a bit with the I've pushed some code in my fork with a test just for the Some highlights:
Would love to hear your thoughts. EDIT: added a commit fixing an import, for some reason when running the |
I've pushed more changes. Some comments:
Now that I have worked out the codemods stuff (and I'm quite happy with it) I'll do an attempt of allowing installation of new packages. Will post here if I make any progress. I'm also not sure If I have to keep spamming this issue or maybe create a PR to avoid notifying everyone? |
Very interesting stuff you have made. I gave up on codemods quite early because it seemed really overcomplicated for what I initially wanted it to be (maybe there is a market for a better tool here @MWhite-22 😃)
Sounds good. If you get it to a working point without it being too complex for contributors to use I would gladly incorporate this into the existing installers, don't know about you @nexxeln?
I don't think anyone minds the "spam", the findings are very interested (I'll admit the initial " Super appreciate the work you have put into this. I don't know if this issue's original point (adding dependencies to an existing project) is something we want to incorporate (I think this is beyond a scaffolding CLI which this is, please confirm @theobr & @nexxeln), but this could definitely be used for the scaffold process! |
Yeah we can definitely include it in the scaffolding, as for the installing deps thing, that can be made into another package? Also this is really exciting stuff! |
Thanks for your kind words!
Yes the codemods literally blew my mind initially 🤯 it feels such a powerful tool though.
My thoughts exactly. IMHO at least it's worth just for conditionally apply changes in the initial installation.
To be honest with you, I'm really interested into looking for a potential solution for this because the starters/CLIs I've used are consumed in a way that your created project becomes a snapshot of that starter/CLIs in that point in time and then you are on your own (it's difficult to add updates/new features added in starter). And personally I've been into situations where I'm maintaining several projects created from the same starter that they require a lot of time to keep updated. Jumping back to our issue at hand, I'll try to break down the next decisions we need to take:
(a) how to trigger new installationsI was thinking that running the This feels like a solution relatively simple to implement and easy to consume, as we would remove the need to have different CLIs or commands, but let me know your thoughts on this. (b) how to detect if a project was created using
|
Hey! Just wanted to give my input on this because I actually built a similar project a while ago called next-gen which has this exact feature. It's currently unmaintained because I don't have the capacity, oh well.. Imo using a library like jscodeshift will be absolutely indispensable for this, the other approaches I tried were way too brittle, It's a bit complicated to get started, but once you get the hang of it it will make the whole process a lot easier. Blitz does a lot of things with jscodeshift, they've written a really nice abstraction for creating your own new recipes as well, you should take a look: https://github.com/blitz-js/blitz/tree/canary/recipes. The nice thing about the jscodeshift approach over templates for scaffolding is that you immensely cut down on combinations of templates. I.e. you won't need a I'm also in the discord, hmu if you have any questions @ochicf . |
Quick updateI've created a new branch in my fork (created from the codemods branch) where I'm handling all of these reinstalls/incremental feature addition. IMHO it would be best to first migrate all existing code to apply codemods, so I wanted to keep my original branch branch with just that work so we can work on that independently. @biowaffeln this is where your knowledge of codemods and the references you gave will be super helpful ❤️ Now I'll quote some parts of my previous comment that I could work on:
Did some basic implementation to store the installation info in Example of generated `package.json`{
"name": "my-t3-app",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"postinstall": "prisma generate"
},
"dependencies": {
// ...
},
"devDependencies": {
// ...
},
"createT3App": {
"version": "5.0.0-beta.7",
"createdAt": "2022-07-18T11:43:13.512Z",
"updatedAt": null,
"packages": {
"nextAuth": {
"installed": true
},
"prisma": {
"installed": true
},
"tailwind": {
"installed": false
},
"trpc": {
"installed": true
}
}
}
} Some notes:
I've done this, but I've kept the const pkg: AvailablePackages = 'tailwind';
const installed = projectPkgJson?.createT3App?.packages?.[pkg]?.installed ?? false;
const installing = packages.includes(pkg);
const inUse = installed || installing; Next stepsI'll continue working on the reinstalls/incremental feature addition. My goal is to "close the circle" and have it implemented for at least a combination of 2 packages (ex: install 1 first, then install the other and expect all the changes to apply). |
I see potential here for a separate package that's usable in other projects as well. We can create a new repo under t3-oss if you're down and have it as a dependency for create-t3-app. That way other projects can use it as well. What do you think? |
That would be so cool. But to be on the safe side maybe I would keep it as a referenced project during the the refactor and make it a separate package afterwards? Not sure if it's the best approach but I'm feeling such a noob with codemods that I feel it's too ambitious to open that package from minute zero 😆 An important thing though: there will be codemods super specific with the templates of this project so maybe those should stay in the project itself (which is closer to the actual files being modified), and leave the external package for helpers/common codemods? |
@juliusmarminge answering #208 (comment) in this issue.
I only applied transform for a single file as a proof of concept Should we maybe create a new issue regarding codemods and regroup there? This way we separate from this, which needs the codemods done. Also in order to properly split the work:
|
List of file changesHere you have a (semi)detailed list of the affected files. I tried to keep simple with symbols and such but... I don't know if it's clear as I'm doing this with my eyes half closed 😴 Wall of text ahead: open at your own risk ☢️ ☠️Notes
Legend:
Changesenv
next-auth
prisma
tailwind
trcp
base
page-studsTLDR: Summary of codemods(you can search the codemod key to see affected files in the wall of text above) [env-1]
[auth-1]
[trpc-1]
[trpc-2]
[pages-1]
[pages-2]
BTW @juliusmarminge I don't have right to modify the GitHub project (cannot add items/tasks to it) |
Now you do |
FWIW, I think we should target the @next branch so we dont do something that will have to be changed in the near future. |
Sorry for the radio silence, I'm currently swamped with work as we are trying to deliver a lot of things before the holidays. On monday we did a call with @biowaffeln to put things in common and he has done a great job and managed to advance a lot with the codemods. He has a lot more experience with those so he will submit a MR from his fork and we can review and test it. @biowaffeln let me know if you need anything from me ;) Once that's up we will have the base we need to keep working on incrementally adding features. |
Finally some update on this. I'm making some progress on moving this project towards using git patches instead of copying and pasting files. I have a partially working project (has tailwind and trpc). If anyone wants to test this, you could run: git clone https://github.com/minsk-dev/create-t3-app-template
cd create-t3-app-template
git checkout pkg/tailwind
git format-patch main -o patches -U1
git checkout pkg/tailwind+trpc
git format-patch main -o patches -U1
git checkout pkg/trpc
git format-patch main -o patches -U1
git checkout main
git checkout -b dev
git apply patches/0001-feat-tailwind-adds-boilerplate.patch
git apply patches/0002-feat-tailwind-adds-dependencies.patch
git apply patches/0003-feat-tailwind-adds-imports.patch
git apply patches/0004-feat-tailwind-updates-index.patch
git apply patches/0001-feat-trpc-adds-boilerplate.patch
git apply patches/0002-feat-trpc-adds-dependencies.patch
git apply patches/0003-feat-trpc-adds-api.patch
git apply patches/0004-feat-trpc-finishes-setup.patch
git apply patches/0005-feat-tailwind-trpc-adds-query.patch
echo "patches" >> .gitignore
git add .
git commit -m "chore: inits project"
pnpm install
pnpm dev Going to create some quality of life for developing with this template, but this is very doable. |
moving this over to voidcoefficient/t3-cli#2. if no one opposes |
Functionality to run create-t3-app in an existing t3 project to add a missing feature, ie. Next Auth, if it wasn't included in original project init.
Adds to the "only use what is needed" axiom while continuing to use t3 boilerplate when enhancements are required.
The text was updated successfully, but these errors were encountered: