Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL on FHIR view runner #1775

Open
wants to merge 320 commits into
base: main
Choose a base branch
from
Open

SQL on FHIR view runner #1775

wants to merge 320 commits into from

Conversation

piotrszul
Copy link
Collaborator

@piotrszul piotrszul commented Jan 16, 2024

This pull request adds the ability to execute SQL on FHIR view definitions against data sources within the library API.

It also includes a comprehensive refactoring that integrates this new functionality with the rest of the code base and makes some changes that will improve performance across all of the different FHIRPath query operations.

Resolves #1438 and #1759.

johngrimes and others added 30 commits September 8, 2023 21:40
…ments to Spark/Catalyst.

Initial concept of annotation fhipath based function definition.
Refactoring of ParserTest and related assertion classes, to work with column based approach (WIP).
Implementation of 'union' and 'forEach' clauses.
Separation of 'v1' FHIRView test from our custom enhanced tests.
Implemented 'getResourceKey()' function.
Implementing 'collection' and singular value selection semantics for DirectSelection.
…ons.

Implementing 'r-keys' functions: getReferenceKey() and getResourceKey().
Cleaning up our extended FhirView tests.
…to allow FHIRView paths like: forEach = 'where(gender='male').name,
…lemented as functions in (StandardFunctions).
# Conflicts:
#	fhirpath/src/main/java/au/csiro/pathling/sql/boundary/DecimalBoundaryFunction.java
# Conflicts:
#	deployment/helm/pom.xml
#	encoders/pom.xml
#	encoders/src/main/java/au/csiro/pathling/config/EncodingConfiguration.java
#	fhir-server/pom.xml
#	fhir-server/src/main/java/au/csiro/pathling/aggregate/AggregateExecutor.java
#	fhir-server/src/main/java/au/csiro/pathling/async/AsyncAspect.java
#	fhir-server/src/main/java/au/csiro/pathling/async/JobProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/caching/EntityTagInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/config/AuthorizationConfiguration.java
#	fhir-server/src/main/java/au/csiro/pathling/config/ServerConfiguration.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/DiagnosticContext.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/ErrorHandlingInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/ErrorReportingInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/extract/ResultProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/ConformanceProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/FhirServer.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/SmartConfigurationInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/search/SearchExecutor.java
#	fhir-server/src/main/java/au/csiro/pathling/security/SecurityConfiguration.java
#	fhir-server/src/test/java/au/csiro/pathling/aggregate/DrillDownBuilderTest.java
#	fhir-server/src/test/java/au/csiro/pathling/extract/ExtractRequestBuilder.java
#	fhir-server/src/test/java/au/csiro/pathling/fhir/EntityTagInterceptorTest.java
#	fhir-server/src/test/java/au/csiro/pathling/fhirpath/parser/ParserTest.java
#	fhir-server/src/test/java/au/csiro/pathling/jmh/SpringBootJmhContext.java
#	fhir-server/src/test/java/au/csiro/pathling/test/IntegrationTestDependencies.java
#	fhirpath/pom.xml
#	fhirpath/src/main/java/au/csiro/pathling/QueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/QueryHelpers.java
#	fhirpath/src/main/java/au/csiro/pathling/aggregate/AggregateQueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/aggregate/AggregateRequest.java
#	fhirpath/src/main/java/au/csiro/pathling/config/StorageConfiguration.java
#	fhirpath/src/main/java/au/csiro/pathling/extract/ExtractQueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/FhirPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Materializable.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/NonLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Numeric.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Referrer.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/ResourcePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Temporal.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/TerminologyUtils.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/UntypedResourcePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/comparison/CodingComparator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/definition/ResourceDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/BooleanPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/CodingPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DatePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DateTimePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DecimalPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ElementDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ElementPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ExtensionPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/IntegerPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/QuantityPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ReferenceExtensionDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ReferencePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/StringPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/TimePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/AggregateFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/Arguments.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/BooleansTestFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/CountFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/EmptyFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ExistsFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ExtensionFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/FirstFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/IifFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NamedFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NamedFunctionInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NotFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/OfTypeFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ResolveFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ReverseResolveFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/SumFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/UntilFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/WhereFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/DesignationFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/DisplayFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/MemberOfFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/PropertyFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/SubsumesFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/TranslateFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/BooleanLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/CodingLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DateLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DateTimeLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DecimalLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/IntegerLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/LiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/NullLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/QuantityLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/StringLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/TimeLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BinaryOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BinaryOperatorInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BooleanOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/CombineOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/Comparable.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/ComparisonOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/DateArithmeticOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/MathOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/MembershipOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/Operator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/PathTraversalInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/PathTraversalOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/InvocationVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/LiteralTermVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/ParserContext.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/TermVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/Visitor.java
#	fhirpath/src/test/java/au/csiro/pathling/UnitTestDependencies.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/CanBeCombinedWithTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/BooleansTestFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/ResolveFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/UntilFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/DesignationFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/PropertyFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/SubsumesFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/TranslateFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/ComparisonOperatorTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/DateArithmeticTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/MathOperatorQuantityTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/MathOperatorTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/QuantityOperatorsPrecisionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/Assertions.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/BaseFhirPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/ElementPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/FhirPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/LiteralPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/ResourcePathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ElementPathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ParserContextBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ResourcePathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/UntypedResourcePathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/helpers/SparkHelpers.java
#	lib/R/pom.xml
#	lib/import/pom.xml
#	lib/js/pom.xml
#	lib/python/Dockerfile
#	lib/python/pom.xml
#	library-api/pom.xml
#	library-api/src/main/java/au/csiro/pathling/library/PathlingContext.java
#	library-api/src/main/java/au/csiro/pathling/library/io/sink/DataSinkBuilder.java
#	library-api/src/main/java/au/csiro/pathling/library/io/source/AbstractSource.java
#	library-api/src/main/java/au/csiro/pathling/library/io/source/QueryableDataSource.java
#	library-api/src/main/java/au/csiro/pathling/library/query/QueryDispatcher.java
#	library-api/src/test/java/au/csiro/pathling/library/PathlingContextTest.java
#	library-runtime/pom.xml
#	pom.xml
#	site/pom.xml
#	terminology/pom.xml
#	terminology/src/main/java/au/csiro/pathling/config/HttpClientCachingConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/HttpClientConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/TerminologyAuthConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/TerminologyConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/terminology/DefaultTerminologyService.java
#	terminology/src/main/java/au/csiro/pathling/terminology/TerminologyServiceFactory.java
#	terminology/src/main/java/au/csiro/pathling/terminology/caching/CachingTerminologyService.java
#	terminology/src/test/java/au/csiro/pathling/test/helpers/TerminologyServiceHelpers.java
#	utilities/pom.xml
#	utilities/src/main/java/au/csiro/pathling/utilities/Lists.java
#	utilities/src/main/java/au/csiro/pathling/validation/ValidationUtils.java
@johngrimes
Copy link
Member

We need to make a corresponding change on this branch for c6b06e7, so that we can properly handle the encoding of arrays of complex elements as null.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request refactoring Improving the design of existing code
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

SQL on FHIR view layer - POC implementation
2 participants