How does LakeSoul store and use unstructured data? Are there any examples?

[dpengpeng]

In the AI+Data scenario,How does LakeSoul store and use unstructured data? Are there any examples?

[xuchen-plus]

We'll release vector indexing feature very soon. See ongoing development: #568 #571 .
We use LSH to compute and store binary encoding for embedding vectors. And vector similarity search can be done using normal SQL together with other filters, e.g.

select v.id, v.attribute,
  calculateHammingDistance(query_embedding, v.embedding) AS hamming_distance
from vector_table v
order by hamming_distance limit 10;

The vector indexing feature is designed for large scale analytical usage for unstructured data on data lake.