Modular architecture

[ilyapuchka]

Currently Sourcery can be used only as a tool, though theoretically it can be used as a dependency to power other tools. We got such requests several times in the past but for now were rejecting them. I'd like to suggest to think about that again because I think that not only Sourcery's parsing functionality can be useful but also its JS and Swift templates. Here are some steps that we will need to make for that:

• Extract parser and composer to a separate framework so that it will provide all parsing functionality.
• Extract JS and Swift templates to separate frameworks
• Extract tests to their own targets, need to remove dependency on Sourcery main target in tests
• Make Sourcery just a command line tool that utilise other frameworks (maybe we'll find a way to conditionally import templates frameworks as plugins)
• Make sure everything works with all dependency managers
• [Optionally] move Sourcery to dedicated organisation so that we can group all those projects. This will also allow us to host forks of Sourcery's own dependencies. Alternatively we can keep everything in one repo and add sub specs, but it will not play very well with Carthage (need to check that and maybe suggest improvement to support building only selected schemes)

[krzysztofzablocki]

Other tools use Sourcery as dependency, there are quite a few that do it, what benefits do you see of using this as a framework rather than a command line tool like they do now with swifttemplates?

Sourcery uses macOS target because embedding swift framework has not been supported properly in command line tools, not sure if that changed yet.

I'm not against us modularizing the project into frameworks so people can use just parsing part of Sourcery, but I'd very much would like to keep all of the frameworks in this repo, I've experienced a mess that managing dependencies across multiple repos can be and I'd rather keep using mono repo here.

[ilyapuchka]

I think it will be more convenient to get parsed data from some sort of API provided by a framework than invoking command line tool and then working with its results in swift templates, if I understand what you mean. The same way we could probably use SourceKitten as command line tool and then parse its output, but using it as a framework is much better.

If they use it as a framework users will also be able to extend it to their needs (if we allow that of course), maybe using different templates engine, or even utilising parsing results for something other than code generation, i.e. displaying types dependencies graph.

Also it will probably push us to cleaning up our test suite and maybe improve some APIs as they will need to be exposed publicly.

Agree with you about managing dependencies from different repos, was there.

[krzysztofzablocki]

templating engines will be pluggable when we do this as our current arch is already pluggable interface so hopefully they will send their extensions upstream to us :)

I remember you wanted to work on extracting Sourcery to a framework @hartbit, would you like to start on this plan ?

[hartbit]

Other tools use Sourcery as dependency, there are quite a few that do it, what benefits do you see of using this as a framework rather than a command line tool like they do now with swifttemplates?

To give some context, I’m working on a tool where I need to parse a source project with Sourcery, and some XML files to generate Swift code based on both the XML files and the Source code of the project. Doing so with swifttemplates would be very complicated/cumbersome.

Sourcery uses macOS target because embedding swift framework has not been supported properly in command line tools, not sure if that changed yet.

It’s possible if the command line tool is built with SwiftPM: it statically links dependencies.

I remember you wanted to work on extracting Sourcery to a framework @hartbit, would you like to start on this plan ?

If we do this piecemeal, I could start by migrating most of the parsing in its own framework and keep the template logic in the command line tool. That can be done in a second step.

[ilyapuchka]

It’s possible if the command line tool is built with SwiftPM: it statically links dependencies.

Another reason for me to look into #305 today

[hectr]

Hi!
I have been digging the issues and pull requests, but I couldn't make my mind on what is the current status of this issue. Is the modular architecture in the project's roadmap?

I share the opinion that being able to use Sourcery as a framework would open many new ways to take advantage of its full potential.

I did a small proof of concept of using Sourcery only for parsing and then loading the result into another program. This setup more or less resembles what you would get from using it as a framework. In the example I wrote I used the information provided by Sourcery to write two reports with metrics of the analyzed sources and a diagram of reference cycles (you can see the code here). The metrics I extracted may be silly and/or not accurate, but I see it as an example of the sort of things that you can easyly do when you use a framework and that would be difficult to achieve only using templates.

[dmsl1805]

Hi.
Sourcery has a great mechanism of parsing metadata. My needs are to use parsing and generation separately in my extension because templates way of generating output data is not enough. I have to do more work between this two processes.
I will be happy to help with it, so want to discuss how it is better to implement.