Skip to content

[Refactor] SerDe files to separate directory #1059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 23, 2025
Merged

[Refactor] SerDe files to separate directory #1059

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
24598e2
serde folder refactor
Oct 23, 2025
187bd9a
serde folder refactor
Oct 23, 2025
2d9c72b
Merge remote-tracking branch 'refs/remotes/origin/awhittier/serde-ref…
Oct 23, 2025
a96d9b2
merge conflict
Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion flink/src/main/scala/ai/chronon/flink/AvroCodecFn.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ import org.slf4j.LoggerFactory
import ai.chronon.api.Extensions.GroupByOps
import ai.chronon.api.{Constants, DataModel, Query, StructType => ChrononStructType}
import ai.chronon.flink.window.TimestampedTile
import ai.chronon.online.{AvroConversions, GroupByServingInfoParsed}
import ai.chronon.online.GroupByServingInfoParsed
import ai.chronon.online.serde.AvroConversions
import ai.chronon.online.KVStore.PutRequest
import org.apache.flink.api.common.functions.RichFlatMapFunction
import org.apache.flink.configuration.Configuration
Expand Down
3 changes: 2 additions & 1 deletion flink/src/main/scala/ai/chronon/flink/FlinkJob.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import ai.chronon.flink.window.{
KeySelector,
TimestampedTile
}
import ai.chronon.online.{GroupByServingInfoParsed, SparkConversions}
import ai.chronon.online.GroupByServingInfoParsed
import ai.chronon.online.serde.SparkConversions
import ai.chronon.online.KVStore.PutRequest
import org.apache.flink.streaming.api.scala.{DataStream, OutputTag, StreamExecutionEnvironment}
import org.apache.spark.sql.Encoder
Expand Down
3 changes: 2 additions & 1 deletion flink/src/main/scala/ai/chronon/flink/SparkExpressionEvalFn.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ package ai.chronon.flink
import org.slf4j.LoggerFactory
import ai.chronon.api.Extensions.{GroupByOps, MetadataOps}
import ai.chronon.api.{Constants, GroupBy, Query, StructType => ChrononStructType}
import ai.chronon.online.{CatalystUtil, SparkConversions}
import ai.chronon.online.CatalystUtil
import ai.chronon.online.serde.SparkConversions
import com.codahale.metrics.ExponentiallyDecayingReservoir
import org.apache.flink.api.common.functions.RichFlatMapFunction
import org.apache.flink.configuration.Configuration
Expand Down
3 changes: 2 additions & 1 deletion flink/src/main/scala/ai/chronon/flink/window/FlinkRowAggregators.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ package ai.chronon.flink.window
import ai.chronon.aggregator.row.RowAggregator
import ai.chronon.api.Extensions.GroupByOps
import ai.chronon.api.{Constants, DataType, GroupBy, Row}
import ai.chronon.online.{ArrayRow, TileCodec}
import ai.chronon.online.TileCodec
import ai.chronon.online.serde.ArrayRow
import org.apache.flink.api.common.functions.AggregateFunction
import org.apache.flink.configuration.Configuration
import org.apache.flink.metrics.Counter
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/CatalystUtil.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package ai.chronon.online
import ai.chronon.api.{DataType, StructType}
import ai.chronon.online.CatalystUtil.{IteratorWrapper, PoolKey, poolMap}
import ai.chronon.online.Extensions.StructTypeOps
import ai.chronon.online.serde.SparkConversions
import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.expressions.UnsafeProjection
import org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/Extensions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package ai.chronon.online

import ai.chronon.api
import ai.chronon.online.serde.{AvroCodec, AvroConversions, SparkConversions}
import org.apache.avro.Schema
import org.apache.spark.sql.types.StructType

Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/Fetcher.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import ai.chronon.online.DerivationUtils.{applyDeriveFunc, buildDerivedFields}
import ai.chronon.online.Fetcher._
import ai.chronon.online.KVStore.GetRequest
import ai.chronon.online.Metrics.Environment
import ai.chronon.online.serde.{AvroCodec, AvroConversions}
import com.google.gson.Gson
import com.timgroup.statsd.Event
import com.timgroup.statsd.Event.AlertType
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/FetcherBase.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import ai.chronon.online.FetcherCache.{BatchResponses, CachedBatchResponse, KvSt
import ai.chronon.online.KVStore.{GetRequest, GetResponse, TimedValue}
import ai.chronon.online.Metrics.Name
import ai.chronon.online.DerivationUtils.{applyDeriveFunc, buildRenameOnlyDerivationFunction}
import ai.chronon.online.serde.AvroConversions
import com.google.gson.Gson

import java.util
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/GroupByServingInfoParsed.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import ai.chronon.aggregator.windowing.SawtoothOnlineAggregator
import ai.chronon.api.Constants.{ReversalField, TimeField}
import ai.chronon.api.Extensions.{GroupByOps, MetadataOps}
import ai.chronon.api._
import ai.chronon.online.serde.{AvroCodec, AvroConversions}
import org.apache.avro.Schema

import ai.chronon.online.DerivationUtils.{DerivationFunc, buildDerivationFunction, buildDerivedFields, timeFields}
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/JoinCodec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package ai.chronon.online
import ai.chronon.api.Extensions.{JoinOps, MetadataOps}
import ai.chronon.api.{DataType, HashUtils, StructField, StructType}
import ai.chronon.online.Fetcher.ResponseWithContext
import ai.chronon.online.serde.{AvroCodec, AvroConversions}
import com.google.gson.Gson

import scala.collection.Seq
Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/MetadataStore.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import ai.chronon.api._
import ai.chronon.online.KVStore.{GetRequest, PutRequest, TimedValue}
import ai.chronon.online.MetadataEndPoint.{ConfByKeyEndPointName, NameByTeamEndPointName}
import ai.chronon.online.Metrics.Name.Exception
import ai.chronon.online.serde.AvroCodec
import com.google.gson.{Gson, GsonBuilder}
import org.apache.thrift.TBase

Expand Down
1 change: 1 addition & 0 deletions online/src/main/scala/ai/chronon/online/TileCodec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package ai.chronon.online

import ai.chronon.aggregator.row.RowAggregator
import ai.chronon.api.{BooleanType, DataType, GroupBy, StructType}
import ai.chronon.online.serde.{AvroCodec, AvroConversions}
import org.apache.avro.generic.GenericData
import ai.chronon.api.Extensions.{AggregationOps, MetadataOps, WindowUtils}

Expand Down
2 changes: 1 addition & 1 deletion ...n/scala/ai/chronon/online/AvroCodec.scala → ...a/ai/chronon/online/serde/AvroCodec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package ai.chronon.online
package ai.chronon.online.serde

import ai.chronon.api.{DataType, Row}
import org.apache.avro.Schema
Expand Down
2 changes: 1 addition & 1 deletion ...a/ai/chronon/online/AvroConversions.scala → ...hronon/online/serde/AvroConversions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package ai.chronon.online
package ai.chronon.online.serde

import ai.chronon.api._
import org.apache.avro.Schema
Expand Down
2 changes: 1 addition & 1 deletion .../ai/chronon/online/SparkConversions.scala → ...ronon/online/serde/SparkConversions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package ai.chronon.online
package ai.chronon.online.serde

import ai.chronon.api
import org.apache.spark.sql.Row
Expand Down
1 change: 1 addition & 0 deletions online/src/test/scala/ai/chronon/online/FetcherCacheTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import ai.chronon.online.Fetcher.Request
import ai.chronon.online.FetcherCache.{BatchIrCache, BatchResponses, CachedMapBatchResponse}
import ai.chronon.online.KVStore.TimedValue
import ai.chronon.online.Metrics.Context
import ai.chronon.online.serde.AvroCodec
import org.junit.Assert.{assertArrayEquals, assertEquals, assertNull, fail}
import org.junit.Test
import org.mockito.Mockito._
Expand Down
1 change: 1 addition & 0 deletions online/src/test/scala/ai/chronon/online/TileCodecTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import ai.chronon.api.{
TimeUnit,
Window
}
import ai.chronon.online.serde.ArrayRow
import org.junit.Assert.assertEquals
import org.junit.Test

Expand Down
3 changes: 2 additions & 1 deletion online/src/test/scala/ai/chronon/online/test/DataStreamBuilderTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ package ai.chronon.online.test

import org.slf4j.LoggerFactory
import ai.chronon.api.{Builders, DataModel, LongType, StringType, StructField, StructType}
import ai.chronon.online.{DataStream, SparkConversions, TopicInfo}
import ai.chronon.online.{DataStream, TopicInfo}
import ai.chronon.online.serde.SparkConversions
import ai.chronon.online.TopicInfo.parse
import org.apache.spark.sql.{DataFrame, Row, SparkSession}
import org.junit.Assert.assertTrue
Expand Down
3 changes: 2 additions & 1 deletion online/src/test/scala/ai/chronon/online/test/ThriftDecodingTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
package ai.chronon.online.test

import ai.chronon.api.{Derivation, Join, StructType}
import ai.chronon.online.{CatalystUtil, SerializableFunction, SparkConversions, TBaseDecoderFactory}
import ai.chronon.online.{CatalystUtil, SerializableFunction, TBaseDecoderFactory}
import ai.chronon.online.serde.SparkConversions
import com.google.gson.Gson
import org.apache.spark.rdd.RDD
import org.apache.spark.sql.Row
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/Analyzer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import ai.chronon.api
import ai.chronon.api.DataModel.{DataModel, Entities, Events}
import ai.chronon.api.Extensions._
import ai.chronon.api.{Accuracy, AggregationPart, Constants, DataType, TimeUnit, Window}
import ai.chronon.online.SparkConversions
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Driver.parseConf
import ai.chronon.spark.Extensions.StructTypeOps
import ai.chronon.spark.catalog.TableUtils
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/BootstrapInfo.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import org.slf4j.LoggerFactory
import ai.chronon.api
import ai.chronon.api.Extensions._
import ai.chronon.api.{Constants, ExternalPart, JoinPart, StructField}
import ai.chronon.online.SparkConversions
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import org.apache.spark.sql.Row
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/Extensions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ package ai.chronon.spark
import org.slf4j.LoggerFactory
import ai.chronon.api
import ai.chronon.api.Constants
import ai.chronon.online.{AvroCodec, AvroConversions, SparkConversions}
import ai.chronon.online.serde.{AvroCodec, AvroConversions, SparkConversions}
import ai.chronon.spark.catalog.TableUtils
import org.apache.avro.Schema
import org.apache.spark.sql.{DataFrame, Row}
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/GroupBy.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import ai.chronon.api
import ai.chronon.api.DataModel.{Entities, Events}
import ai.chronon.api.Extensions._
import ai.chronon.api.{Accuracy, Constants, DataModel, ParametricMacro}
import ai.chronon.online.{RowWrapper, SparkConversions}
import ai.chronon.online.serde.{RowWrapper, SparkConversions}
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import org.apache.spark.rdd.RDD
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/GroupByUpload.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ import ai.chronon.api
import ai.chronon.api.{Accuracy, Constants, DataModel, GroupByServingInfo, QueryUtils, ThriftJsonCodec}
import ai.chronon.api.Extensions.{GroupByOps, MetadataOps, SourceOps}
import ai.chronon.online.Extensions.ChrononStructTypeOps
import ai.chronon.online.{GroupByServingInfoParsed, Metrics, SparkConversions}
import ai.chronon.online.{GroupByServingInfoParsed, Metrics}
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import org.apache.spark.SparkEnv
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/Join.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ import ai.chronon.api
import ai.chronon.api.DataModel.Entities
import ai.chronon.api.Extensions._
import ai.chronon.api._
import ai.chronon.online.{DerivationUtils, SparkConversions}
import ai.chronon.online.DerivationUtils
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.JoinUtils._
import ai.chronon.spark.catalog.TableUtils
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/KvRdd.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ package ai.chronon.spark

import org.slf4j.LoggerFactory
import ai.chronon.api
import ai.chronon.online.{AvroCodec, AvroConversions, SparkConversions}
import ai.chronon.online.serde.{AvroCodec, AvroConversions, SparkConversions}
import ai.chronon.spark.Extensions._
import org.apache.avro.generic.GenericData
import org.apache.spark.rdd.RDD
Expand Down
1 change: 1 addition & 0 deletions spark/src/main/scala/ai/chronon/spark/LogFlattenerJob.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import org.apache.spark.rdd.RDD
import org.apache.spark.sql.catalyst.expressions.GenericRow
import org.apache.spark.sql.functions.col
import org.apache.spark.sql.{DataFrame, Dataset, Row, SparkSession}
import ai.chronon.online.serde.{AvroConversions, SparkConversions}

import java.util.Base64
import scala.+:
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/LoggingSchema.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
package ai.chronon.spark

import ai.chronon.api.{HashUtils, StructField, StructType}
import ai.chronon.online.{AvroCodec, JoinCodec}
import ai.chronon.online.JoinCodec
import ai.chronon.online.serde.AvroCodec
import com.google.gson.Gson

import scala.util.ScalaJavaConversions.MapOps
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/stats/CompareBaseJob.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ package ai.chronon.spark.stats

import org.slf4j.LoggerFactory
import ai.chronon.api._
import ai.chronon.online.{SparkConversions, _}
import ai.chronon.online._
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.{TimedKvRdd}
import ai.chronon.spark.catalog.TableUtils
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/stats/CompareJob.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ import ai.chronon.api
import ai.chronon.api.{Constants, PartitionSpec}
import ai.chronon.api.DataModel.Events
import ai.chronon.api.Extensions._
import ai.chronon.online.{DataMetrics, SparkConversions}
import ai.chronon.online.DataMetrics
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.catalog.TableUtils
import ai.chronon.spark.stats.CompareJob.getJoinKeys
import ai.chronon.spark.{Analyzer, PartitionRange, StagingQuery, TimedKvRdd}
Expand Down
3 changes: 2 additions & 1 deletion spark/src/main/scala/ai/chronon/spark/stats/CompareMetrics.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ package ai.chronon.spark.stats
import ai.chronon.aggregator.row.RowAggregator
import ai.chronon.api.Extensions.{AggregationPartOps, WindowUtils}
import ai.chronon.api._
import ai.chronon.online.{DataMetrics, SparkConversions}
import ai.chronon.online.DataMetrics
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.catalog.TableUtils
import ai.chronon.spark.{Comparison, TimedKvRdd}
import org.apache.spark.rdd.RDD
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/stats/StatsCompute.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import ai.chronon.aggregator.row.{RowAggregator, StatsGenerator}
import ai.chronon.api
import ai.chronon.api.Extensions._
import ai.chronon.spark.Extensions._
import ai.chronon.online.SparkConversions
import ai.chronon.online.serde.SparkConversions
import com.yahoo.memory.Memory
import com.yahoo.sketches.kll.KllFloatsSketch
import org.apache.spark.sql.functions.col
Expand Down
2 changes: 1 addition & 1 deletion spark/src/main/scala/ai/chronon/spark/stats/SummaryJob.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
package ai.chronon.spark.stats

import org.slf4j.LoggerFactory
import ai.chronon.online.SparkConversions
import ai.chronon.online.serde.SparkConversions
import ai.chronon.aggregator.row.StatsGenerator
import ai.chronon.api.Extensions._
import ai.chronon.api._
Expand Down
1 change: 1 addition & 0 deletions spark/src/main/scala/ai/chronon/spark/streaming/GroupBy.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import ai.chronon.spark.{EncoderUtil, GenericRowHandler}
import com.google.gson.Gson
import org.apache.spark.sql._
import org.apache.spark.sql.streaming.{DataStreamWriter, StreamingQuery, Trigger}
import ai.chronon.online.serde.{AvroConversions, SparkConversions}

import java.time.format.DateTimeFormatter
import java.time.{Instant, ZoneId, ZoneOffset}
Expand Down
1 change: 1 addition & 0 deletions spark/src/main/scala/ai/chronon/spark/streaming/JoinSourceRunner.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import org.apache.spark.sql.streaming.{DataStreamWriter, Trigger}
import org.apache.spark.sql.types.{BooleanType, LongType, StructField, StructType}
import org.apache.spark.sql.{DataFrame, Dataset, Encoder, Encoders, Row, SparkSession}
import org.slf4j.LoggerFactory
import ai.chronon.online.serde.{AvroConversions, SparkConversions}

import java.time.format.DateTimeFormatter
import java.time.{Instant, ZoneId, ZoneOffset}
Expand Down
3 changes: 2 additions & 1 deletion spark/src/test/scala/ai/chronon/spark/test/ChainingFetcherTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ import ai.chronon.api.Constants.ChrononMetadataKey
import ai.chronon.api.Extensions.{JoinOps, MetadataOps}
import ai.chronon.api._
import ai.chronon.online.Fetcher.Request
import ai.chronon.online.{MetadataStore, SparkConversions}
import ai.chronon.online.MetadataStore
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import ai.chronon.spark.{Join => _, _}
Expand Down
2 changes: 1 addition & 1 deletion spark/src/test/scala/ai/chronon/spark/test/DataFrameGen.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ package ai.chronon.spark.test

import ai.chronon.aggregator.test.{CStream, Column, RowsWithSchema}
import ai.chronon.api.{Constants, LongType, StringType}
import ai.chronon.online.SparkConversions
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.catalog.TableUtils
import org.apache.spark.rdd.RDD
import org.apache.spark.sql.catalyst.expressions.GenericRow
Expand Down
3 changes: 2 additions & 1 deletion spark/src/test/scala/ai/chronon/spark/test/FetcherTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ import ai.chronon.api.Constants.ChrononMetadataKey
import ai.chronon.api.Extensions.{JoinOps, MetadataOps}
import ai.chronon.api._
import ai.chronon.online.Fetcher.{Request, Response, StatsRequest}
import ai.chronon.online.{JavaRequest, LoggableResponseBase64, MetadataStore, SparkConversions}
import ai.chronon.online.{JavaRequest, LoggableResponseBase64, MetadataStore}
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import ai.chronon.spark.stats.ConsistencyJob
Expand Down
2 changes: 1 addition & 1 deletion spark/src/test/scala/ai/chronon/spark/test/GroupByTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import ai.chronon.api.{
TimeUnit,
Window
}
import ai.chronon.online.{RowWrapper, SparkConversions}
import ai.chronon.online.serde.{RowWrapper, SparkConversions}
import ai.chronon.spark.Extensions._
import ai.chronon.spark._
import ai.chronon.spark.catalog.TableUtils
Expand Down
3 changes: 2 additions & 1 deletion spark/src/test/scala/ai/chronon/spark/test/InMemoryStream.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ package ai.chronon.spark.test

import org.slf4j.LoggerFactory
import ai.chronon.api.{Constants, GroupBy, StructType}
import ai.chronon.online.{AvroConversions, Mutation, SparkConversions, TileCodec}
import ai.chronon.online.{Mutation, TileCodec}
import ai.chronon.online.serde.{AvroConversions, SparkConversions}
import ai.chronon.online.Extensions.StructTypeOps
import ai.chronon.spark.GenericRowHandler
import ai.chronon.spark.catalog.TableUtils
Expand Down
1 change: 1 addition & 0 deletions spark/src/test/scala/ai/chronon/spark/test/MockApi.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import ai.chronon.api.Extensions.{GroupByOps, SourceOps}
import ai.chronon.api.{Constants, StructType}
import ai.chronon.online.Fetcher.Response
import ai.chronon.online._
import ai.chronon.online.serde.{AvroConversions, SparkConversions}
import ai.chronon.spark.Extensions._
import ai.chronon.spark.catalog.TableUtils
import org.apache.avro.Schema
Expand Down
3 changes: 2 additions & 1 deletion spark/src/test/scala/ai/chronon/spark/test/OnlineUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ import ai.chronon.spark.streaming.{GroupBy, JoinSourceRunner}
import ai.chronon.spark.stats.SummaryJob
import org.apache.spark.sql.streaming.Trigger
import ai.chronon.api.Extensions.{GroupByOps, MetadataOps, SourceOps}
import ai.chronon.online.{AvroConversions, KVStore, TileCodec}
import ai.chronon.online.{KVStore, TileCodec}
import ai.chronon.online.serde.AvroConversions
import ai.chronon.spark.catalog.TableUtils
import org.apache.spark.sql.SparkSession

Expand Down
1 change: 1 addition & 0 deletions spark/src/test/scala/ai/chronon/spark/test/SchemaEvolutionTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import ai.chronon.api.Extensions.MetadataOps
import ai.chronon.api._
import ai.chronon.online.Fetcher.Request
import ai.chronon.online._
import ai.chronon.online.serde.SparkConversions
import ai.chronon.spark.Extensions.DataframeOps
import ai.chronon.spark.catalog.TableUtils
import ai.chronon.spark.{LogFlattenerJob, LoggingSchema, SparkSessionBuilder}
Expand Down
2 changes: 1 addition & 1 deletion spark/src/test/scala/ai/chronon/spark/test/StatsComputeTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import org.slf4j.LoggerFactory
import ai.chronon.aggregator.row.StatsGenerator
import ai.chronon.aggregator.test.Column
import ai.chronon.api._
import ai.chronon.online.SparkConversions.toChrononSchema
import ai.chronon.online.serde.SparkConversions.toChrononSchema
import ai.chronon.spark.Extensions._
import ai.chronon.spark.SparkSessionBuilder
import ai.chronon.spark.catalog.TableUtils
Expand Down
Loading