Support for multiple Mikro-ORM instances with context (RequestContext / AsyncLocalStorage)

[hilkeheremans]

Is your feature request related to a problem? Please describe.
I need to use Mikro-ORM to connect to different database instances, so I have multiple Mikro ORM instances running in my NestJS application. (One is located within an npm dependency, the other is located in the NestJS application directly).

At the moment it seems I cannot use the Mikro-ORM instance in the npm dependency, because the RequestContext helper only supports a single Mikro-ORM entity manager -- more specifically, whatever Mikro ORM EM is used in the RequestContext.create call.

Describe the solution you'd like
I'd like to find a solution that allows multiple instances of entity manager, belonging to different mikro orm instances, to coexist, without them interfering with one another.

A potential way to do this is to combine the following solutions:

• allow Mikro-ORM/entitymanager instances to be named, making them 'aware' of their own name;
• using the name for the entity manager instances to fetch the proper entity manager from context;
• creating new versions of RequestContext and a wrapper for AsyncLocalStorage that supports storing/fetching multiple entity managers

Describe alternatives you've considered
I've considered rolling the two database instances into one, but that will likely cause trouble for us down the road since we need to be able to scale them independently.

Additional context

• I have a commit that makes EntityManager 'name-aware' over at hilkeheremans@3c84e9a. Note: I could not verify this via tests or add new tests, since I can't get the tests to run properly. Unclear what I need to do to fully set up the schema in the db's of the docker containers.

• I believe making RequestContext/AsyncLocalStorage compatible with multiple Entity Managers may require a slightly more fundamental change. The way this context is created now, eg via RequestContext.create() forces you to specify the entity manager you want to fork. Is this still the road we want to take, or is an alternative method, ie letting the user of RequestContext specify a set of root entity managers initially in code, all of which can then be forked on each new request? This would also simplify the API.

• Either way, these new versions of RequestContext/AsyncLocalStorage would be breaking changes, so it would probably make sense to create these new versions alongside the existing ones, so users can opt-in to them at will.

Thanks a bunch for your input and let me know what I am missing.

[B4nan]

What about this:

• EM will have an optional name
• the name will be passed as a parameter to the context callback
• you will implement custom context method that will pick the right EM from current context holder (let it be domain api or ALS)
• we can have additional version of the RequestContext helper to reduce boilerplate, but the original class will stay as is to maintain BC

ok, I should have checked your commit sooner, I see you basically did exactly that :] that sounds like correct way to me

Something like this could work for the context helper:

import domain, { Domain } from 'domain';
import { EntityManager } from '../EntityManager';
import { Dictionary } from '../typings';

export type MultiORMDomain = Domain & { __mikro_orm_context?: MultiRequestContext };

/**
 * For node 14 and above it is suggested to use `AsyncLocalStorage` instead,
 * @see https://mikro-orm.io/docs/async-local-storage/
 */
export class MultiRequestContext {

  private static counter = 1;
  readonly id = MultiRequestContext.counter++;

  constructor(readonly map: Map<string, EntityManager>) { }

  /**
   * Creates new RequestContext instance and runs the code inside its domain.
   */
  static create(map: Map<string, EntityManager>, next: (...args: any[]) => void): void {
    const forks = new Map<string, EntityManager>();
    map.forEach((em, name) => forks.set(name, em.fork(true, true)));
    const context = new MultiRequestContext(forks);
    const d = domain.create() as MultiORMDomain;
    d.__mikro_orm_context = context;
    d.run(next);
  }

  /**
   * Creates new RequestContext instance and runs the code inside its domain.
   * Async variant, when the `next` handler needs to be awaited (like in Koa).
   */
  static async createAsync(map: Map<string, EntityManager>, next: (...args: any[]) => Promise<void>): Promise<void> {
    const forks = new Map<string, EntityManager>();
    map.forEach((em, name) => forks.set(name, em.fork(true, true)));
    const context = new MultiRequestContext(forks);
    const d = domain.create() as MultiORMDomain;
    d.__mikro_orm_context = context;
    await new Promise((resolve, reject) => {
      d.run(() => next().then(resolve).catch(reject));
    });
  }

  /**
   * Returns current RequestContext (if available).
   */
  static currentRequestContext(): MultiRequestContext | undefined {
    const active = (domain as Dictionary).active as MultiORMDomain;
    return active ? active.__mikro_orm_context : undefined;
  }

  /**
   * Returns current EntityManager (if available).
   */
  static getEntityManager(name: string): EntityManager | undefined {
    const context = MultiRequestContext.currentRequestContext();
    return context ? context.map.get(name) : undefined;
  }

}

[B4nan]

Any updates on this? I am planning to ship 4.1 soon, will be happy to include MultiRequestContext if you can confirm it does what we need.

[hilkeheremans]

Not yet, I have other priorities this week. It may take a while.

[stalniy]

Does mikro supports multiple instances without RequestContext? I saw that MetadataStorage has global metadata property where every EntityMetadata is stored (if I understood it correctly).

If I use EntitySchema + custom init function to create connection, I guess it should work, right?

[B4nan]

RequestContext is only about automatic handling of contexts, you don't need it if you fork the EM manually. It is also handy when you use DI and have singletons of repositories, but as long as you use em.fork() and work with that (and get repositories from that fork via em.getRepository()), there is now need to RequestContext helper.

[stalniy]

Sounds good!

what about global static metadata field on MetadataStorage? and what if I want to have entities with the same name on different connections?

[B4nan]

Thats only for decorators, so with EntitySchema it should be all good.

Also as long as you compile your TS via tsc, the decorators will use current path to entity in the global storage key, so its possible to have multiple entities with same name in different files (as long as you do not try to use them both in one ORM instance) even with decorators.

[B4nan]

v4.3 will allow specifying contextName in the config (name was already taken by connection name), which will be used to get the right EM in RequestContext. You can now pass array of EMs in the create method.

Let me know if it works, will be happy to reopen.

[hilkeheremans]

Wow, thanks. I was just about to pick this up again and just noticed you closed it. I'll add a test ticket to my workload to test this soon.