Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTLP/GRPC Exporter Implementation #272

Merged
merged 37 commits into from
May 5, 2021
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
dc64b04
Adding Proto to opentelemetry-php
riticksingh Nov 11, 2020
dfe7b26
Updated the script for generating proto folder in root
riticksingh Nov 13, 2020
7205339
Moved shell file in script folder
riticksingh Nov 13, 2020
dc6fe75
Removed opentelemetry-proto
riticksingh Nov 14, 2020
09a405a
Bug fixed
riticksingh Nov 15, 2020
3444fb9
added docker-compose.proto.yaml
riticksingh Nov 17, 2020
ffae565
Update docker-compose.proto.yaml
riticksingh Nov 17, 2020
1cc76d4
Updated README.md
riticksingh Nov 17, 2020
7a3fdcd
Merge remote-tracking branch 'upstream/master'
riticksingh Nov 21, 2020
d536ec8
OtlpGrpc
riticksingh Nov 21, 2020
40bf96f
Added Request-response
riticksingh Nov 22, 2020
e02b931
Example Updated
riticksingh Nov 24, 2020
b691299
Updated Span Converter
riticksingh Nov 26, 2020
912584d
small fixes
riticksingh Dec 16, 2020
9bb160f
modified: SpanConvertor.php
riticksingh Dec 16, 2020
cf54582
(WIP): Partially working OTLP/GRPC Exporter
SeanHood Mar 4, 2021
354853f
Fixed traceid and spanid hacks; Now supports Events
SeanHood Mar 8, 2021
7f9530c
Use localhost:4317 for the default host; + cleanup
SeanHood Mar 12, 2021
8c63f52
Merge remote-tracking branch 'origin/main' into otlp-grpc
SeanHood Mar 12, 2021
580acf8
Split code between Exporter.php and SpanConverter.php; Beginnings of …
SeanHood Mar 17, 2021
b8469e6
More progress on tests, array's now correctly get converted. Also ran…
SeanHood Mar 19, 2021
2c5ccf4
* Ironed out passing in headers. Could not figure out how best to val…
SeanHood Mar 31, 2021
844ebf2
Add (failing) test for converting SDK Span to OTLP Span
SeanHood Apr 6, 2021
621175c
Further progress on Tests for the Exporter
SeanHood Apr 6, 2021
5c54d21
SpanConverter now translates attrs on Events
SeanHood Apr 9, 2021
49fa87d
Add opentelemtry-proto files
SeanHood Apr 13, 2021
996b473
Ran `make style' on proto. Do we want this?
SeanHood Apr 13, 2021
5dd3dd9
Few changes in this one..
SeanHood Apr 13, 2021
03a767a
Fix some of the complaints from static analysis
SeanHood Apr 21, 2021
40ce35f
Merge branch 'main' of github.com:SeanHood/opentelemetry-php into otl…
SeanHood Apr 21, 2021
0eb6cfd
Revert "Ran `make style' on proto. Do we want this?"
SeanHood Apr 21, 2021
469afde
make style
SeanHood Apr 21, 2021
06b68f7
Static analysis fixes
SeanHood Apr 22, 2021
1b2c91a
Hopefully this adds the grpc extension
SeanHood Apr 22, 2021
b46f981
Hopefully this adds the grpc extension
SeanHood Apr 24, 2021
a0fd792
Some Psalm fixes
SeanHood Apr 26, 2021
f68a7f1
Added a few comments; disable testing of InstrumentationLibrary until…
SeanHood May 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@
"guzzlehttp/guzzle": "^6.5",
"psr/http-client": "^1.0",
"php-http/guzzle6-adapter": "^2.0",
"endclothing/prometheus_client_php": "^1.0"
"endclothing/prometheus_client_php": "^1.0",
"grpc/grpc": "^1.30",
"google/protobuf": "^v3.3.0"

},
"authors": [
{
Expand All @@ -30,7 +33,15 @@
"OpenTelemetry\\Context\\": "Context",
"OpenTelemetry\\": "api",
"OpenTelemetry\\Sdk\\": "sdk",
"OpenTelemetry\\Contrib\\": "contrib"
"OpenTelemetry\\Contrib\\": "contrib",
"Opentelemetry\\Proto\\Collector\\Trace\\V1\\": "proto/Opentelemetry/Proto/Collector/Trace/V1",
"Opentelemetry\\Proto\\Trace\\V1\\":"proto/Opentelemetry/Proto/Trace/V1",
"Opentelemetry\\Proto\\Common\\V1\\":"proto/Opentelemetry/Proto/Common/V1",
"Opentelemetry\\Proto\\Resource\\V1\\":"proto/Opentelemetry/Proto/Resource/V1",
"GPBMetadata\\Opentelemetry\\Proto\\Collector\\Trace\\V1\\": "proto/GPBMetadata/Opentelemetry/Proto/Collector/Trace/V1",
"GPBMetadata\\Opentelemetry\\Proto\\Trace\\V1\\":"proto/GPBMetadata/Opentelemetry/Proto/Trace/V1",
"GPBMetadata\\Opentelemetry\\Proto\\Common\\V1\\":"proto/GPBMetadata/Opentelemetry/Proto/Common/V1",
"GPBMetadata\\Opentelemetry\\Proto\\Resource\\V1\\":"proto/GPBMetadata/Opentelemetry/Proto/Resource/V1"
}
},
"autoload-dev": {
Expand Down
285 changes: 285 additions & 0 deletions contrib/OtlpGrpc/Exporter.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,285 @@
<?php

declare(strict_types=1);
namespace OpenTelemetry\Contrib\OtlpGrpc;


use grpc;
use OpenTelemetry\Sdk\Trace;
use OpenTelemetry\Trace as API;
use OpenTelemetry\Trace\Span;
use Opentelemetry\Proto;
use Opentelemetry\Proto\Collector\Trace\V1;
use Opentelemetry\Proto\Common\V1\InstrumentationLibrary;
use Opentelemetry\Proto\Common\V1\KeyValue;
use Opentelemetry\Proto\Common\V1\AnyValue;
use Opentelemetry\Proto\Trace\V1\Span as CollectorSpan;
use Opentelemetry\Proto\Trace\V1\Span\SpanKind;
use Opentelemetry\Proto\Trace\V1\InstrumentationLibrarySpans;
use Opentelemetry\Proto\Trace\V1\Status\StatusCode;
use Opentelemetry\Proto\Trace\V1\Status;

class Exporter implements Trace\Exporter
{
/**
* @var string
*/
private $endpointURL;

/**
* @var string
*/
private $protocol;

/**
* @var string
*/
private $insecure;

/**
* @var string
*/
private $certificateFile;

/**
* @var array
*/
private $headers;

/**
* @var string
*/
private $compression;

/**
* @var int
*/
private $timeout;
/**
* @var SpanConverter
*/
private $spanConverter;

/**
* @var bool
*/
private $running = true;

/**
* @var ClientInterface
*/

private $client;

/**
* OTLP GRPC Exporter Constructor
* @param string $serviceName
*/
public function __construct(
$serviceName,
ClientInterface $client=null
) {

// Set default values based on presence of env variable
SeanHood marked this conversation as resolved.
Show resolved Hide resolved
$this->endpointURL = getenv('OTEL_EXPORTER_OTLP_ENDPOINT') ?: 'host.docker.internal:4317';
//$this->endpointURL = getenv('OTEL_EXPORTER_OTLP_ENDPOINT') ?: 'api.honeycomb.io:443';
$this->protocol = getenv('OTEL_EXPORTER_OTLP_PROTOCOL') ?: 'grpc';
$this->insecure = getenv('OTEL_EXPORTER_OTLP_INSECURE') ?: 'false';
$this->certificateFile = getenv('OTEL_EXPORTER_OTLP_CERTIFICATE') ?: 'none';
$this->headers[] = getenv('OTEL_EXPORTER_OTLP_HEADERS') ?: 'none';
$this->compression = getenv('OTEL_EXPORTER_OTLP_COMPRESSION') ?: 'none';
$this->timeout =(int) getenv('OTEL_EXPORTER_OTLP_TIMEOUT') ?: 10;


$opts = [
SeanHood marked this conversation as resolved.
Show resolved Hide resolved
'credentials' => Grpc\ChannelCredentials::createInsecure(),
//'credentials' => Grpc\ChannelCredentials::createSsl(),
'update_metadata' => function() {
return [
'x-honeycomb-team' => ['xxx'],
'x-honeycomb-dataset' => ['xxx'],
];
},
];


$this->client = $client ?? new V1\TraceServiceClient($this->endpointURL, $opts);

}

/**
* Exports the provided Span data via the OTLP protocol
*
* @param iterable<API\Span> $spans Array of Spans
* @return int return code, defined on the Exporter interface
*/
public function export(iterable $spans): int
{
if (!$this->running) {
return Exporter::FAILED_NOT_RETRYABLE;
}

if (empty($spans)) {
return Trace\Exporter::SUCCESS;
}

$convertedSpans = [];
foreach ($spans as $span) {
array_push($convertedSpans, $this->as_otlp_span($span));
}


$il = new InstrumentationLibrary([
'name' => 'otel-php',
'version' => '0.0.1'
]);

$ilspans = [];
foreach($convertedSpans as $convertedSpan) {
$ilspan = new InstrumentationLibrarySpans([
'instrumentation_library' => $il,
'spans' => [$convertedSpan]
]);

array_push($ilspans, $ilspan);
}


$resourcespans = new Proto\Trace\V1\ResourceSpans([
'instrumentation_library_spans' => $ilspans
]);


$request= new V1\ExportTraceServiceRequest();
$request->setResourceSpans([$resourcespans]);


list($response, $status) = $this->client->Export($request)->wait();
if ($status->code !== Grpc\STATUS_OK) {
echo "ERROR: " . $status->code . ", " . $status->details . PHP_EOL;
}


// TODO: Make this work
//echo $response->getMessage() . PHP_EOL;

// if ($response->getStatusCode() >= 400 && $response->getStatusCode() < 500) {
// return Trace\Exporter::FAILED_NOT_RETRYABLE;
// }

// if ($response->getStatusCode() >= 500 && $response->getStatusCode() < 600) {
// return Trace\Exporter::FAILED_RETRYABLE;
// }

return Trace\Exporter::SUCCESS;
}

public function as_otlp_key_value($key, $value): KeyValue {
return new KeyValue([
'key' => $key,
'value' => $this->as_otlp_any_value($value)
]);
}

public function as_otlp_any_value($value): AnyValue
{
$result = new AnyValue();

switch (true) {
case is_array($value):
$result->setArrayValue($value);
break;
case is_int($value):
$result->setIntValue($value);
break;
case is_bool($value):
$result->setBoolValue($value);
break;
case is_double($value):
$result->setDoubleValue($value);
break;
case is_string($value):
$result->setStringValue($value);
break;
}

return $result;
}

public function as_otlp_span_kind($kind): int
{

switch ($kind) {
case 0: return SpanKind::SPAN_KIND_INTERNAL;
case 1: return SpanKind::SPAN_KIND_CLIENT;
case 2: return SpanKind::SPAN_KIND_SERVER;
case 3: return SpanKind::SPAN_KIND_PRODUCER;
case 4: return SpanKind::SPAN_KIND_CONSUMER;
}

return SpanKind::SPAN_KIND_UNSPECIFIED;
}

// ["trace_id":protected]=> string(32) "f8ebf54644f65fae5e98c5dd84b9d196"
// ["span_id":protected]=> string(16) "b90a8323cfd7cd66"
public function as_otlp_span(Span $span): CollectorSpan
{

$duration_ns = (($span->getEnd() - $span->getStart()));
$end_timestamp = ($span->getStartEpochTimestamp() + $duration_ns);

$row = [
// TODO: Fix this, I was getting 32, and 16 byte ids sent to the collector which it didn't like
SeanHood marked this conversation as resolved.
Show resolved Hide resolved
// fudged it with this substr for now, it's possibly an easy fix
'trace_id' => substr($span->getContext()->getTraceId(), 0, 16),
'span_id' => substr($span->getContext()->getSpanId(), 0, 8),
'parent_span_id' => $span->getParent() ? $span->getParent()->getSpanId() : null,
// 'localEndpoint' => [
// 'serviceName' => $this->serviceName,
// ],
'name' => $span->getSpanName(),
'start_time_unix_nano' => $span->getStartEpochTimestamp(),
'end_time_unix_nano' => $end_timestamp,
'kind' => $this->as_otlp_span_kind($span->getSpanKind()),
// 'trace_state' => $span->getContext()
// 'events' =>
// 'links' =>
];


foreach ($span->getEvents() as $event) {
if (!array_key_exists('annotations', $row)) {
$row['annotations'] = [];
}
$row['annotations'][] = [
'timestamp' => (int) ($event->getTimestamp() / 1e3), // RealtimeClock in microseconds
'value' => $event->getName(),
];
}

foreach ($span->getAttributes() as $k => $v) {
if (!array_key_exists('attributes', $row)) {
$row['attributes'] = [];
}
array_push($row['attributes'], $this->as_otlp_key_value($k, $v->getValue()));

}

if (!array_key_exists('status', $row)) {
$proto_status = StatusCode::STATUS_CODE_OK;
if ($span->getStatus()->getCanonicalStatusCode() === "ERROR") {
$proto_status = StatusCode::STATUS_CODE_ERROR;
}
$status=new Status();
$row['status']=$status->setCode($proto_status)->setMessage("Description");
}

return new CollectorSpan($row);

}
public function shutdown(): void
{
$this->running = false;
}

}
Loading