dgraphloader not loading schema quickly, or client should expose a way to wait for schema update to finish

[srh]

Suppose you do this:

$ ~/go/bin/dgraphloader -r ~/1million.rdf.gz -s data/21million.schema 

Processing data/21million.schema

Processing /home/srh/1million.rdf.gz
[Request:      1] Total RDFs done:    72024 RDFs per second:   35878 Time Elapse
[Request:      1] Total RDFs done:    72024 RDFs per second:   17973 Time Elapse
[Request:      1] Total RDFs done:    72024 RDFs per second:   11989 Time Elapse
[Request:      1] Total RDFs done:    72024 RDFs per second:    8995 Time Elapse
[Request:      1] Total RDFs done:    72024 RDFs per second:    7197 Time Elapse
^C 10s 

Then when I run curl localhost:8080/query -XPOST -d 'schema { }' I get back {"schema":null}.

You would think that it would have the schema in place before loading the data.

It looks like the cause of this is that in client/mutations.go we have AddSchema being defined as

func (d *Dgraph) AddSchema(s protos.SchemaUpdate) error {
	if err := checkSchema(s); err != nil {
		return err
	}
	d.schema <- s
	return nil
}

It just pushes the schema onto a channel.

Is the reason for this that because schema updates iterate over the db, we want to make them out-of-band? It looks like there's no way in the client to wait for a particular schema update to complete.

[manishrjain]

It used to wait. @janardhan1993 ?

[srh]

It seems like it does

	// wait for schema changes to be done before starting mutations
	time.Sleep(1 * time.Second)

which I guess wasn't enough.

[srh]

As I discussed with @janardhan1993 maybe this is more of a feature request. It seems like it would be nice if dgraphloader supported waiting for the schema update to be applied before loading data. Or it would be useful in general if clients could call AddSchema and then wait for the result, because subsequent queries might need to use indexes it specifies.

[manishrjain]

We can add an API, SetSchemaBlocking, which can wait until the schema is applied. And then, the user can start sending the data after this has returned.

[janardhan1993]

We set schema wait until it is set and then only start loading data.