Skip to content

Commit

Permalink
Faster point in polygon / GeoJSON updates
Browse files Browse the repository at this point in the history
The big change is that the GeoJSON package has been completely
rewritten to fix a few of geometry calculation bugs, increase
performance, and to better follow the GeoJSON spec RFC 7946.

GeoJSON updates

- A LineString now requires at least two points.
- All json members, even foreign, now persist with the object.
- The bbox member persists too but is no longer used for geometry
  calculations. This is change in behavior. Previously Tile38 would
  treat the bbox as the object's physical rectangle.
- Corrections to geometry intersects and within calculations.

Faster spatial queries

- The performance of Point-in-polygon and object intersect operations
  are greatly improved for complex polygons and line strings. It went
  from O(n) to roughly O(log n).
- The same for all collection types with many children, including
  FeatureCollection, GeometryCollection, MultiPoint, MultiLineString,
  and MultiPolygon.

Codebase changes

- The pkg directory has been renamed to internal
- The GeoJSON internal package has been moved to a seperate repo at
  https://github.com/tidwall/geojson. It's now vendored.

Please look out for higher memory usage for datasets using complex
shapes. A complex shape is one that has 64 or more points. For these
shapes it's expected that there will be increase of least 54 bytes per
point.
  • Loading branch information
tidwall committed Oct 13, 2018
1 parent 5ad27e7 commit 6257ddb
Show file tree
Hide file tree
Showing 370 changed files with 35,941 additions and 42,242 deletions.
53 changes: 53 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 5 additions & 9 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,11 @@
# name = "github.com/x/y"
# version = "2.4.0"

required = ["github.com/tidwall/lotsa"]
required = [
"github.com/tidwall/lotsa",
"github.com/mmcloughlin/geohash",
"github.com/tidwall/geojson"
]

[[constraint]]
branch = "master"
Expand Down Expand Up @@ -50,10 +54,6 @@ required = ["github.com/tidwall/lotsa"]
branch = "master"
name = "github.com/streadway/amqp"

[[constraint]]
name = "github.com/stretchr/testify"
version = "1.1.4"

[[constraint]]
branch = "master"
name = "github.com/tidwall/btree"
Expand Down Expand Up @@ -82,10 +82,6 @@ required = ["github.com/tidwall/lotsa"]
name = "github.com/tidwall/sjson"
version = "1.0.0"

[[constraint]]
branch = "master"
name = "github.com/tidwall/tinyqueue"

[[constraint]]
branch = "master"
name = "golang.org/x/crypto"
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<p align="center">
<a href="http://tile38.com"><img
src="/pkg/assets/logo1500.png"
src="/internal/assets/logo1500.png"
width="200" height="200" border="0" alt="Tile38"></a>
</p>
<p align="center">
Expand All @@ -14,11 +14,11 @@ Tile38 is an open source (MIT licensed), in-memory geolocation data store, spati

<p align="center">
<i>This README is quick start document. You can find detailed documentation at <a href="http://tile38.com">http://tile38.com</a>.</i><br><br>
<a href="#searching"><img src="/pkg/assets/search-nearby.png" alt="Nearby" border="0" width="120" height="120"></a>
<a href="#searching"><img src="/pkg/assets/search-within.png" alt="Within" border="0" width="120" height="120"></a>
<a href="#searching"><img src="/pkg/assets/search-intersects.png" alt="Intersects" border="0" width="120" height="120"></a>
<a href="http://tile38.com/topics/geofencing"><img src="/pkg/assets/geofence.gif" alt="Geofencing" border="0" width="120" height="120"></a>
<a href="http://tile38.com/topics/roaming-geofences"><img src="/pkg/assets/roaming.gif" alt="Roaming Geofences" border="0" width="120" height="120"></a>
<a href="#searching"><img src="/internal/assets/search-nearby.png" alt="Nearby" border="0" width="120" height="120"></a>
<a href="#searching"><img src="/internal/assets/search-within.png" alt="Within" border="0" width="120" height="120"></a>
<a href="#searching"><img src="/internal/assets/search-intersects.png" alt="Intersects" border="0" width="120" height="120"></a>
<a href="http://tile38.com/topics/geofencing"><img src="/internal/assets/geofence.gif" alt="Geofencing" border="0" width="120" height="120"></a>
<a href="http://tile38.com/topics/roaming-geofences"><img src="/internal/assets/roaming.gif" alt="Roaming Geofences" border="0" width="120" height="120"></a>
</p>

## Features
Expand Down Expand Up @@ -134,19 +134,19 @@ To set a field when an object already exists:

Tile38 has support to search for objects and points that are within or intersects other objects. All object types can be searched including Polygons, MultiPolygons, GeometryCollections, etc.

<img src="/pkg/assets/search-within.png" width="200" height="200" border="0" alt="Search Within" align="left">
<img src="/internal/assets/search-within.png" width="200" height="200" border="0" alt="Search Within" align="left">

#### Within
WITHIN searches a collection for objects that are fully contained inside a specified bounding area.
<BR CLEAR="ALL">

<img src="/pkg/assets/search-intersects.png" width="200" height="200" border="0" alt="Search Intersects" align="left">
<img src="/internal/assets/search-intersects.png" width="200" height="200" border="0" alt="Search Intersects" align="left">

#### Intersects
INTERSECTS searches a collection for objects that intersect a specified bounding area.
<BR CLEAR="ALL">

<img src="/pkg/assets/search-nearby.png" width="200" height="200" border="0" alt="Search Nearby" align="left">
<img src="/internal/assets/search-nearby.png" width="200" height="200" border="0" alt="Search Nearby" align="left">

#### Nearby
NEARBY searches a collection for objects that intersect a specified radius.
Expand All @@ -167,7 +167,7 @@ NEARBY searches a collection for objects that intersect a specified radius.

## Geofencing

<img src="/pkg/assets/geofence.gif" width="200" height="200" border="0" alt="Geofence animation" align="left">
<img src="/internal/assets/geofence.gif" width="200" height="200" border="0" alt="Geofence animation" align="left">
A <a href="https://en.wikipedia.org/wiki/Geo-fence">geofence</a> is a virtual boundary that can detect when an object enters or exits the area. This boundary can be a radius, bounding box, or a polygon. Tile38 can turn any standard search into a geofence monitor by adding the FENCE keyword to the search.

*Tile38 also allows for [Webhooks](http://tile38.com/commands/sethook) to be assigned to Geofences.*
Expand Down
4 changes: 2 additions & 2 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ if [ "$NOLINK" != "1" ]; then
fi

# generate the core package
pkg/core/gen.sh
core/gen.sh

# build and store objects into original directory.
CGO_ENABLED=0 go build -ldflags "$LDFLAGS -extldflags '-static'" -o "$OD/tile38-server" cmd/tile38-server/*.go
Expand All @@ -178,7 +178,7 @@ if [ "$1" == "test" ]; then
}
trap testend EXIT
cd tests && go test && cd ..
go test $(go list ./... | grep -v /vendor/ | grep -v /tests)
go test $(go list ./... | grep -v /vendor/ | grep -v /tests | grep -v /pkg/geojson_bak)
fi

# cover if requested
Expand Down
2 changes: 1 addition & 1 deletion cmd/tile38-benchmark/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
"time"

"github.com/tidwall/redbench"
"github.com/tidwall/tile38/pkg/core"
"github.com/tidwall/tile38/core"
)

var (
Expand Down
4 changes: 2 additions & 2 deletions cmd/tile38-cli/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ import (
"github.com/peterh/liner"
"github.com/tidwall/gjson"
"github.com/tidwall/resp"
"github.com/tidwall/tile38/pkg/client"
"github.com/tidwall/tile38/pkg/core"
"github.com/tidwall/tile38/core"
"github.com/tidwall/tile38/internal/client"
)

func userHomeDir() string {
Expand Down
10 changes: 4 additions & 6 deletions cmd/tile38-server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,12 @@ import (
"sync"
"syscall"

"github.com/tidwall/tile38/core"
"github.com/tidwall/tile38/internal/controller"
"github.com/tidwall/tile38/internal/hservice"
"github.com/tidwall/tile38/internal/log"
"golang.org/x/net/context"

"google.golang.org/grpc"

"github.com/tidwall/tile38/pkg/controller"
"github.com/tidwall/tile38/pkg/core"
"github.com/tidwall/tile38/pkg/hservice"
"github.com/tidwall/tile38/pkg/log"
)

var (
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
142 changes: 142 additions & 0 deletions internal/clip/clip.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
package clip

import (
"github.com/tidwall/geojson"
"github.com/tidwall/geojson/geometry"
)

// Clip clips the contents of a geojson object and return
func Clip(obj geojson.Object, clipper geojson.Object) (clipped geojson.Object) {
switch obj := obj.(type) {
case *geojson.Point:
return clipPoint(obj, clipper)
case *geojson.Rect:
return clipRect(obj, clipper)
case *geojson.LineString:
return clipLineString(obj, clipper)
case *geojson.Polygon:
return clipPolygon(obj, clipper)
case *geojson.Feature:
return clipFeature(obj, clipper)
case geojson.Collection:
return clipCollection(obj, clipper)
}
return obj
}

// clipSegment is Cohen-Sutherland Line Clipping
// https://www.cs.helsinki.fi/group/goa/viewing/leikkaus/lineClip.html
func clipSegment(seg geometry.Segment, rect geometry.Rect) (
res geometry.Segment, rejected bool,
) {
startCode := getCode(rect, seg.A)
endCode := getCode(rect, seg.B)
if (startCode | endCode) == 0 {
// trivially accept
res = seg
} else if (startCode & endCode) != 0 {
// trivially reject
rejected = true
} else if startCode != 0 {
// start is outside. get new start.
newStart := intersect(rect, startCode, seg.A, seg.B)
res, rejected =
clipSegment(geometry.Segment{A: newStart, B: seg.B}, rect)
} else {
// end is outside. get new end.
newEnd := intersect(rect, endCode, seg.A, seg.B)
res, rejected = clipSegment(geometry.Segment{A: seg.A, B: newEnd}, rect)
}
return
}

// clipRing is Sutherland-Hodgman Polygon Clipping
// https://www.cs.helsinki.fi/group/goa/viewing/leikkaus/intro2.html
func clipRing(ring []geometry.Point, bbox geometry.Rect) (
resRing []geometry.Point,
) {
if len(ring) < 4 {
// under 4 elements this is not a polygon ring!
return
}
var edge uint8
var inside, prevInside bool
var prev geometry.Point
for edge = 1; edge <= 8; edge *= 2 {
prev = ring[len(ring)-2]
prevInside = (getCode(bbox, prev) & edge) == 0
for _, p := range ring {
inside = (getCode(bbox, p) & edge) == 0
if prevInside && inside {
// Staying inside
resRing = append(resRing, p)
} else if prevInside && !inside {
// Leaving
resRing = append(resRing, intersect(bbox, edge, prev, p))
} else if !prevInside && inside {
// Entering
resRing = append(resRing, intersect(bbox, edge, prev, p))
resRing = append(resRing, p)
} else {
// Staying outside
}
prev, prevInside = p, inside
}
if len(resRing) > 0 && resRing[0] != resRing[len(resRing)-1] {
resRing = append(resRing, resRing[0])
}
ring, resRing = resRing, []geometry.Point{}
if len(ring) == 0 {
break
}
}
resRing = ring
return
}

func getCode(bbox geometry.Rect, point geometry.Point) (code uint8) {
code = 0

if point.X < bbox.Min.X {
code |= 1 // left
} else if point.X > bbox.Max.X {
code |= 2 // right
}

if point.Y < bbox.Min.Y {
code |= 4 // bottom
} else if point.Y > bbox.Max.Y {
code |= 8 // top
}

return
}

func intersect(bbox geometry.Rect, code uint8, start, end geometry.Point) (
new geometry.Point,
) {
if (code & 8) != 0 { // top
new = geometry.Point{
X: start.X + (end.X-start.X)*(bbox.Max.Y-start.Y)/(end.Y-start.Y),
Y: bbox.Max.Y,
}
} else if (code & 4) != 0 { // bottom
new = geometry.Point{
X: start.X + (end.X-start.X)*(bbox.Min.Y-start.Y)/(end.Y-start.Y),
Y: bbox.Min.Y,
}
} else if (code & 2) != 0 { //right
new = geometry.Point{
X: bbox.Max.X,
Y: start.Y + (end.Y-start.Y)*(bbox.Max.X-start.X)/(end.X-start.X),
}
} else if (code & 1) != 0 { // left
new = geometry.Point{
X: bbox.Min.X,
Y: start.Y + (end.Y-start.Y)*(bbox.Min.X-start.X)/(end.X-start.X),
}
} else { // should not call intersect with the zero code
}

return
}
Loading

0 comments on commit 6257ddb

Please sign in to comment.