1
0
Fork 0

Added support for Haystack tracing

This commit is contained in:
Alex Antonov 2019-05-08 17:14:04 -05:00 committed by Traefiker Bot
parent 681892148e
commit 9cf6827ccc
274 changed files with 38070 additions and 13436 deletions

View file

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [2018] Expedia Group.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View file

@ -0,0 +1,356 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"fmt"
"os"
"os/signal"
"time"
"github.com/opentracing/opentracing-go/ext"
"github.com/opentracing/opentracing-go/log"
)
/*Dispatcher dispatches the span object*/
type Dispatcher interface {
Name() string
Dispatch(span *_Span)
DispatchProtoSpan(span *Span)
Close()
SetLogger(logger Logger)
}
/*InMemoryDispatcher implements the Dispatcher interface*/
type InMemoryDispatcher struct {
spans []*_Span
logger Logger
}
/*NewInMemoryDispatcher creates a new in memory dispatcher*/
func NewInMemoryDispatcher() Dispatcher {
return &InMemoryDispatcher{}
}
/*Name gives the Dispatcher name*/
func (d *InMemoryDispatcher) Name() string {
return "InMemoryDispatcher"
}
/*SetLogger sets the logger to use*/
func (d *InMemoryDispatcher) SetLogger(logger Logger) {
d.logger = logger
}
/*Dispatch dispatches the span object*/
func (d *InMemoryDispatcher) Dispatch(span *_Span) {
d.spans = append(d.spans, span)
}
/*DispatchProtoSpan dispatches proto span object*/
func (d *InMemoryDispatcher) DispatchProtoSpan(span *Span) {
/* not implemented */
}
/*Close down the inMemory dispatcher*/
func (d *InMemoryDispatcher) Close() {
d.spans = nil
}
/*FileDispatcher file dispatcher*/
type FileDispatcher struct {
fileHandle *os.File
logger Logger
}
/*NewFileDispatcher creates a new file dispatcher*/
func NewFileDispatcher(filename string) Dispatcher {
fd, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
panic(err)
}
return &FileDispatcher{
fileHandle: fd,
}
}
/*Name gives the Dispatcher name*/
func (d *FileDispatcher) Name() string {
return "FileDispatcher"
}
/*SetLogger sets the logger to use*/
func (d *FileDispatcher) SetLogger(logger Logger) {
d.logger = logger
}
/*Dispatch dispatches the span object*/
func (d *FileDispatcher) Dispatch(span *_Span) {
_, err := d.fileHandle.WriteString(span.String() + "\n")
if err != nil {
panic(err)
}
}
/*DispatchProtoSpan dispatches proto span object*/
func (d *FileDispatcher) DispatchProtoSpan(span *Span) {
/* not implemented */
}
/*Close down the file dispatcher*/
func (d *FileDispatcher) Close() {
err := d.fileHandle.Close()
if err != nil {
panic(err)
}
}
/*RemoteDispatcher dispatcher, client can be grpc or http*/
type RemoteDispatcher struct {
client RemoteClient
timeout time.Duration
logger Logger
spanChannel chan *Span
}
/*NewHTTPDispatcher creates a new haystack-agent dispatcher*/
func NewHTTPDispatcher(url string, timeout time.Duration, headers map[string]string, maxQueueLength int) Dispatcher {
dispatcher := &RemoteDispatcher{
client: NewHTTPClient(url, headers, timeout),
timeout: timeout,
spanChannel: make(chan *Span, maxQueueLength),
}
go startListener(dispatcher)
return dispatcher
}
/*NewDefaultHTTPDispatcher creates a new http dispatcher*/
func NewDefaultHTTPDispatcher() Dispatcher {
return NewHTTPDispatcher("http://haystack-collector/span", 3*time.Second, make(map[string](string)), 1000)
}
/*NewAgentDispatcher creates a new haystack-agent dispatcher*/
func NewAgentDispatcher(host string, port int, timeout time.Duration, maxQueueLength int) Dispatcher {
dispatcher := &RemoteDispatcher{
client: NewGrpcClient(host, port, timeout),
timeout: timeout,
spanChannel: make(chan *Span, maxQueueLength),
}
go startListener(dispatcher)
return dispatcher
}
/*NewDefaultAgentDispatcher creates a new haystack-agent dispatcher*/
func NewDefaultAgentDispatcher() Dispatcher {
return NewAgentDispatcher("haystack-agent", 35000, 3*time.Second, 1000)
}
func startListener(dispatcher *RemoteDispatcher) {
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt, os.Kill)
for {
select {
case sp := <-dispatcher.spanChannel:
dispatcher.client.Send(sp)
case <-signals:
break
}
}
}
/*Name gives the Dispatcher name*/
func (d *RemoteDispatcher) Name() string {
return "RemoteDispatcher"
}
/*SetLogger sets the logger to use*/
func (d *RemoteDispatcher) SetLogger(logger Logger) {
d.logger = logger
d.client.SetLogger(logger)
}
/*Dispatch dispatches the span object*/
func (d *RemoteDispatcher) Dispatch(span *_Span) {
s := &Span{
TraceId: span.context.TraceID,
SpanId: span.context.SpanID,
ParentSpanId: span.context.ParentID,
ServiceName: span.ServiceName(),
OperationName: span.OperationName(),
StartTime: span.startTime.UnixNano() / int64(time.Microsecond),
Duration: span.duration.Nanoseconds() / int64(time.Microsecond),
Tags: d.tags(span),
Logs: d.logs(span),
}
d.spanChannel <- s
}
/*DispatchProtoSpan dispatches the proto span object*/
func (d *RemoteDispatcher) DispatchProtoSpan(s *Span) {
d.spanChannel <- s
}
func (d *RemoteDispatcher) logs(span *_Span) []*Log {
var spanLogs []*Log
for _, lg := range span.logs {
spanLogs = append(spanLogs, &Log{
Timestamp: lg.Timestamp.UnixNano() / int64(time.Microsecond),
Fields: d.logFieldsToTags(lg.Fields),
})
}
return spanLogs
}
func (d *RemoteDispatcher) logFieldsToTags(fields []log.Field) []*Tag {
var spanTags []*Tag
for _, field := range fields {
spanTags = append(spanTags, ConvertToProtoTag(field.Key(), field.Value()))
}
return spanTags
}
func (d *RemoteDispatcher) tags(span *_Span) []*Tag {
var spanTags []*Tag
for _, tag := range span.tags {
spanTags = append(spanTags, ConvertToProtoTag(tag.Key, tag.Value))
}
return spanTags
}
/*Close down the file dispatcher*/
func (d *RemoteDispatcher) Close() {
err := d.client.Close()
if err != nil {
d.logger.Error("Fail to close the haystack-agent dispatcher %v", err)
}
}
/*ConvertToProtoTag converts to proto tag*/
func ConvertToProtoTag(key string, value interface{}) *Tag {
switch v := value.(type) {
case string:
return &Tag{
Key: key,
Myvalue: &Tag_VStr{
VStr: value.(string),
},
Type: Tag_STRING,
}
case int:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(int)),
},
Type: Tag_LONG,
}
case int32:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(int32)),
},
Type: Tag_LONG,
}
case int16:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(int16)),
},
Type: Tag_LONG,
}
case int64:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: value.(int64),
},
Type: Tag_LONG,
}
case uint16:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(uint16)),
},
Type: Tag_LONG,
}
case uint32:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(uint32)),
},
Type: Tag_LONG,
}
case uint64:
return &Tag{
Key: key,
Myvalue: &Tag_VLong{
VLong: int64(value.(uint64)),
},
Type: Tag_LONG,
}
case float32:
return &Tag{
Key: key,
Myvalue: &Tag_VDouble{
VDouble: float64(value.(float32)),
},
Type: Tag_DOUBLE,
}
case float64:
return &Tag{
Key: key,
Myvalue: &Tag_VDouble{
VDouble: value.(float64),
},
Type: Tag_DOUBLE,
}
case bool:
return &Tag{
Key: key,
Myvalue: &Tag_VBool{
VBool: value.(bool),
},
Type: Tag_BOOL,
}
case []byte:
return &Tag{
Key: key,
Myvalue: &Tag_VBytes{
VBytes: value.([]byte),
},
Type: Tag_BINARY,
}
case ext.SpanKindEnum:
return &Tag{
Key: key,
Myvalue: &Tag_VStr{
VStr: string(value.(ext.SpanKindEnum)),
},
Type: Tag_STRING,
}
default:
panic(fmt.Errorf("unknown format %v", v))
}
}

View file

@ -0,0 +1,37 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
/*Logger defines a new logger interface*/
type Logger interface {
Info(format string, v ...interface{})
Error(format string, v ...interface{})
Debug(format string, v ...interface{})
}
/*NullLogger does nothing*/
type NullLogger struct{}
/*Error prints the error message*/
func (logger NullLogger) Error(format string, v ...interface{}) {}
/*Info prints the info message*/
func (logger NullLogger) Info(format string, v ...interface{}) {}
/*Debug prints the info message*/
func (logger NullLogger) Debug(format string, v ...interface{}) {}

View file

@ -0,0 +1,193 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"fmt"
"net/url"
"strings"
"github.com/opentracing/opentracing-go"
)
/*Propagator defines the interface for injecting and extracing the SpanContext from the carrier*/
type Propagator interface {
// Inject takes `SpanContext` and injects it into `carrier`
Inject(ctx *SpanContext, carrier interface{}) error
// Extract `SpanContext` from the `carrier`
Extract(carrier interface{}) (*SpanContext, error)
}
/*Codex defines the interface for encoding and decoding the propagated data*/
type Codex interface {
Encode(value string) string
Decode(value string) string
}
/*DefaultCodex is a no op*/
type DefaultCodex struct{}
/*Encode a no-op for encoding the value*/
func (c DefaultCodex) Encode(value string) string { return value }
/*Decode a no-op for decoding the value*/
func (c DefaultCodex) Decode(value string) string { return value }
/*URLCodex encodes decodes a url*/
type URLCodex struct{}
/*Encode a no-op for encoding the value*/
func (c URLCodex) Encode(value string) string { return url.QueryEscape(value) }
/*Decode a no-op for decoding the value*/
func (c URLCodex) Decode(value string) string {
decoded, err := url.QueryUnescape(value)
if err == nil {
return decoded
}
return ""
}
/*PropagatorOpts defines the options need by a propagator*/
type PropagatorOpts struct {
TraceIDKEYName string
SpanIDKEYName string
ParentSpanIDKEYName string
BaggagePrefixKEYName string
}
var defaultPropagatorOpts = PropagatorOpts{}
var defaultCodex = DefaultCodex{}
/*TraceIDKEY returns the trace id key in the propagator*/
func (p *PropagatorOpts) TraceIDKEY() string {
if p.TraceIDKEYName != "" {
return p.TraceIDKEYName
}
return "Trace-ID"
}
/*SpanIDKEY returns the span id key in the propagator*/
func (p *PropagatorOpts) SpanIDKEY() string {
if p.SpanIDKEYName != "" {
return p.SpanIDKEYName
}
return "Span-ID"
}
/*ParentSpanIDKEY returns the parent span id key in the propagator*/
func (p *PropagatorOpts) ParentSpanIDKEY() string {
if p.ParentSpanIDKEYName != "" {
return p.ParentSpanIDKEYName
}
return "Parent-ID"
}
/*BaggageKeyPrefix returns the baggage key prefix*/
func (p *PropagatorOpts) BaggageKeyPrefix() string {
if p.BaggagePrefixKEYName != "" {
return p.BaggagePrefixKEYName
}
return "Baggage-"
}
/*TextMapPropagator implements Propagator interface*/
type TextMapPropagator struct {
opts PropagatorOpts
codex Codex
}
/*Inject injects the span context in the carrier*/
func (p *TextMapPropagator) Inject(ctx *SpanContext, carrier interface{}) error {
textMapWriter, ok := carrier.(opentracing.TextMapWriter)
if !ok {
return opentracing.ErrInvalidCarrier
}
textMapWriter.Set(p.opts.TraceIDKEY(), ctx.TraceID)
textMapWriter.Set(p.opts.SpanIDKEY(), ctx.SpanID)
textMapWriter.Set(p.opts.ParentSpanIDKEY(), ctx.ParentID)
ctx.ForeachBaggageItem(func(key, value string) bool {
textMapWriter.Set(fmt.Sprintf("%s%s", p.opts.BaggageKeyPrefix(), key), p.codex.Encode(ctx.Baggage[key]))
return true
})
return nil
}
/*Extract the span context from the carrier*/
func (p *TextMapPropagator) Extract(carrier interface{}) (*SpanContext, error) {
textMapReader, ok := carrier.(opentracing.TextMapReader)
if !ok {
return nil, opentracing.ErrInvalidCarrier
}
baggageKeyLowerCasePrefix := strings.ToLower(p.opts.BaggageKeyPrefix())
traceIDKeyLowerCase := strings.ToLower(p.opts.TraceIDKEY())
spanIDKeyLowerCase := strings.ToLower(p.opts.SpanIDKEY())
parentSpanIDKeyLowerCase := strings.ToLower(p.opts.ParentSpanIDKEY())
traceID := ""
spanID := ""
parentSpanID := ""
baggage := make(map[string]string)
err := textMapReader.ForeachKey(func(k, v string) error {
lcKey := strings.ToLower(k)
if strings.HasPrefix(lcKey, baggageKeyLowerCasePrefix) {
keySansPrefix := lcKey[len(p.opts.BaggageKeyPrefix()):]
baggage[keySansPrefix] = p.codex.Decode(v)
} else if lcKey == traceIDKeyLowerCase {
traceID = v
} else if lcKey == spanIDKeyLowerCase {
spanID = v
} else if lcKey == parentSpanIDKeyLowerCase {
parentSpanID = v
}
return nil
})
if err != nil {
return nil, err
}
return &SpanContext{
TraceID: traceID,
SpanID: spanID,
ParentID: parentSpanID,
Baggage: baggage,
IsExtractedContext: true,
}, nil
}
/*NewDefaultTextMapPropagator returns a default text map propagator*/
func NewDefaultTextMapPropagator() *TextMapPropagator {
return NewTextMapPropagator(defaultPropagatorOpts, defaultCodex)
}
/*NewTextMapPropagator returns a text map propagator*/
func NewTextMapPropagator(opts PropagatorOpts, codex Codex) *TextMapPropagator {
return &TextMapPropagator{
opts: opts,
codex: codex,
}
}

View file

@ -0,0 +1,165 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"bytes"
"fmt"
"io/ioutil"
"net/http"
"time"
"github.com/golang/protobuf/proto"
context "golang.org/x/net/context"
grpc "google.golang.org/grpc"
)
/*RemoteClient remote client*/
type RemoteClient interface {
Send(span *Span)
Close() error
SetLogger(logger Logger)
}
/*GrpcClient grpc client*/
type GrpcClient struct {
conn *grpc.ClientConn
client SpanAgentClient
timeout time.Duration
logger Logger
}
/*NewGrpcClient returns a new grpc client*/
func NewGrpcClient(host string, port int, timeout time.Duration) *GrpcClient {
targetHost := fmt.Sprintf("%s:%d", host, port)
conn, err := grpc.Dial(targetHost, grpc.WithInsecure())
if err != nil {
panic(fmt.Sprintf("fail to connect to agent with error: %v", err))
}
return &GrpcClient{
conn: conn,
client: NewSpanAgentClient(conn),
timeout: timeout,
}
}
/*Send a proto span to grpc server*/
func (c *GrpcClient) Send(span *Span) {
ctx, cancel := context.WithTimeout(context.Background(), c.timeout)
defer cancel()
result, err := c.client.Dispatch(ctx, span)
if err != nil {
c.logger.Error("Fail to dispatch to haystack-agent with error %v", err)
} else if result.GetCode() != DispatchResult_SUCCESS {
c.logger.Error(fmt.Sprintf("Fail to dispatch to haystack-agent with error code: %d, message :%s", result.GetCode(), result.GetErrorMessage()))
} else {
c.logger.Debug(fmt.Sprintf("span [%v] has been successfully dispatched to haystack", span))
}
}
/*Close the grpc client*/
func (c *GrpcClient) Close() error {
return c.conn.Close()
}
/*SetLogger sets the logger*/
func (c *GrpcClient) SetLogger(logger Logger) {
c.logger = logger
}
/*HTTPClient a http client*/
type HTTPClient struct {
url string
headers map[string]string
client *http.Client
logger Logger
}
/*NewHTTPClient returns a new http client*/
func NewHTTPClient(url string, headers map[string]string, timeout time.Duration) *HTTPClient {
httpClient := &http.Client{
Timeout: timeout,
}
return &HTTPClient{
url: url,
headers: headers,
client: httpClient,
}
}
/*Send a proto span to http server*/
func (c *HTTPClient) Send(span *Span) {
serializedBytes, marshalErr := proto.Marshal(span)
if marshalErr != nil {
c.logger.Error("Fail to serialize the span to proto bytes, error=%v", marshalErr)
return
}
postRequest, requestErr := http.NewRequest(http.MethodPost, c.url, bytes.NewReader(serializedBytes))
if requestErr != nil {
c.logger.Error("Fail to create request for posting span to haystack server, error=%v", requestErr)
return
}
if c.headers != nil {
for k, v := range c.headers {
postRequest.Header.Add(k, v)
}
}
resp, err := c.client.Do(postRequest)
if err != nil {
c.logger.Error("Fail to dispatch to haystack http server, error=%v", err)
}
defer func() {
closeErr := resp.Body.Close()
if closeErr != nil {
/* do nothing */
}
}()
respBytes, respErr := ioutil.ReadAll(resp.Body)
if respErr != nil {
c.logger.Error("Fail to read the http response from haystack server, error=%v", respErr)
return
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
c.logger.Error("Fail to dispatch the span to haystack http server with statusCode=%d , payload=%s", resp.StatusCode, string(respBytes))
} else {
c.logger.Debug(fmt.Sprintf("span [%v] has been successfully dispatched to haystack, response=%s", span, string(respBytes)))
}
}
/*Close the http client*/
func (c *HTTPClient) Close() error {
return nil
}
/*SetLogger sets the logger*/
func (c *HTTPClient) SetLogger(logger Logger) {
c.logger = logger
}

View file

@ -0,0 +1,164 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"encoding/json"
"time"
"github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/log"
)
/*_Span implements opentracing.Span*/
type _Span struct {
tracer *Tracer
context *SpanContext
operationName string
// startTime is the timestamp indicating when the span began, with microseconds precision.
startTime time.Time
// duration returns duration of the span with microseconds precision.
duration time.Duration
tags []opentracing.Tag
logs []opentracing.LogRecord
}
// SetOperationName sets or changes the operation name.
func (span *_Span) SetOperationName(operationName string) opentracing.Span {
span.operationName = operationName
return span
}
// SetTag implements SetTag() of opentracing.Span
func (span *_Span) SetTag(key string, value interface{}) opentracing.Span {
span.tags = append(span.tags,
opentracing.Tag{
Key: key,
Value: value,
})
return span
}
// LogFields implements opentracing.Span API
func (span *_Span) LogFields(fields ...log.Field) {
log := opentracing.LogRecord{
Fields: fields,
Timestamp: time.Now(),
}
span.logs = append(span.logs, log)
}
// LogKV implements opentracing.Span API
func (span *_Span) LogKV(alternatingKeyValues ...interface{}) {
fields, err := log.InterleavedKVToFields(alternatingKeyValues...)
if err != nil {
span.LogFields(log.Error(err), log.String("function", "LogKV"))
return
}
span.LogFields(fields...)
}
// LogEvent implements opentracing.Span API
func (span *_Span) LogEvent(event string) {
span.Log(opentracing.LogData{Event: event})
}
// LogEventWithPayload implements opentracing.Span API
func (span *_Span) LogEventWithPayload(event string, payload interface{}) {
span.Log(opentracing.LogData{Event: event, Payload: payload})
}
// Log implements opentracing.Span API
func (span *_Span) Log(ld opentracing.LogData) {
span.logs = append(span.logs, ld.ToLogRecord())
}
// SetBaggageItem implements SetBaggageItem() of opentracing.SpanContext
func (span *_Span) SetBaggageItem(key, value string) opentracing.Span {
span.context = span.context.WithBaggageItem(key, value)
span.LogFields(log.String("event", "baggage"), log.String("payload", key), log.String("payload", value))
return span
}
// BaggageItem implements BaggageItem() of opentracing.SpanContext
func (span *_Span) BaggageItem(key string) string {
return span.context.Baggage[key]
}
// Finish implements opentracing.Span API
func (span *_Span) Finish() {
span.FinishWithOptions(opentracing.FinishOptions{})
}
// FinishWithOptions implements opentracing.Span API
func (span *_Span) FinishWithOptions(options opentracing.FinishOptions) {
if options.FinishTime.IsZero() {
options.FinishTime = span.tracer.timeNow()
}
span.duration = options.FinishTime.Sub(span.startTime)
if options.LogRecords != nil {
span.logs = append(span.logs, options.LogRecords...)
}
for _, ld := range options.BulkLogData {
span.logs = append(span.logs, ld.ToLogRecord())
}
span.tracer.DispatchSpan(span)
}
// Context implements opentracing.Span API
func (span *_Span) Context() opentracing.SpanContext {
return span.context
}
/*Tracer returns the tracer*/
func (span *_Span) Tracer() opentracing.Tracer {
return span.tracer
}
/*OperationName allows retrieving current operation name*/
func (span *_Span) OperationName() string {
return span.operationName
}
/*ServiceName returns the name of the service*/
func (span *_Span) ServiceName() string {
return span.tracer.serviceName
}
func (span *_Span) String() string {
data, err := json.Marshal(map[string]interface{}{
"traceId": span.context.TraceID,
"spanId": span.context.SpanID,
"parentSpanId": span.context.ParentID,
"operationName": span.OperationName(),
"serviceName": span.ServiceName(),
"tags": span.Tags(),
"logs": span.logs,
})
if err != nil {
panic(err)
}
return string(data)
}
func (span *_Span) Tags() []opentracing.Tag {
return span.tags
}

View file

@ -0,0 +1,433 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: span.proto
package haystack
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
// TagType denotes the type of a Tag's value.
type Tag_TagType int32
const (
Tag_STRING Tag_TagType = 0
Tag_DOUBLE Tag_TagType = 1
Tag_BOOL Tag_TagType = 2
Tag_LONG Tag_TagType = 3
Tag_BINARY Tag_TagType = 4
)
var Tag_TagType_name = map[int32]string{
0: "STRING",
1: "DOUBLE",
2: "BOOL",
3: "LONG",
4: "BINARY",
}
var Tag_TagType_value = map[string]int32{
"STRING": 0,
"DOUBLE": 1,
"BOOL": 2,
"LONG": 3,
"BINARY": 4,
}
func (x Tag_TagType) String() string {
return proto.EnumName(Tag_TagType_name, int32(x))
}
func (Tag_TagType) EnumDescriptor() ([]byte, []int) {
return fileDescriptor_fc5f2b88b579999f, []int{2, 0}
}
// Span represents a unit of work performed by a service.
type Span struct {
TraceId string `protobuf:"bytes,1,opt,name=traceId,proto3" json:"traceId,omitempty"`
SpanId string `protobuf:"bytes,2,opt,name=spanId,proto3" json:"spanId,omitempty"`
ParentSpanId string `protobuf:"bytes,3,opt,name=parentSpanId,proto3" json:"parentSpanId,omitempty"`
ServiceName string `protobuf:"bytes,4,opt,name=serviceName,proto3" json:"serviceName,omitempty"`
OperationName string `protobuf:"bytes,5,opt,name=operationName,proto3" json:"operationName,omitempty"`
StartTime int64 `protobuf:"varint,6,opt,name=startTime,proto3" json:"startTime,omitempty"`
Duration int64 `protobuf:"varint,7,opt,name=duration,proto3" json:"duration,omitempty"`
Logs []*Log `protobuf:"bytes,8,rep,name=logs,proto3" json:"logs,omitempty"`
Tags []*Tag `protobuf:"bytes,9,rep,name=tags,proto3" json:"tags,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Span) Reset() { *m = Span{} }
func (m *Span) String() string { return proto.CompactTextString(m) }
func (*Span) ProtoMessage() {}
func (*Span) Descriptor() ([]byte, []int) {
return fileDescriptor_fc5f2b88b579999f, []int{0}
}
func (m *Span) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Span.Unmarshal(m, b)
}
func (m *Span) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Span.Marshal(b, m, deterministic)
}
func (m *Span) XXX_Merge(src proto.Message) {
xxx_messageInfo_Span.Merge(m, src)
}
func (m *Span) XXX_Size() int {
return xxx_messageInfo_Span.Size(m)
}
func (m *Span) XXX_DiscardUnknown() {
xxx_messageInfo_Span.DiscardUnknown(m)
}
var xxx_messageInfo_Span proto.InternalMessageInfo
func (m *Span) GetTraceId() string {
if m != nil {
return m.TraceId
}
return ""
}
func (m *Span) GetSpanId() string {
if m != nil {
return m.SpanId
}
return ""
}
func (m *Span) GetParentSpanId() string {
if m != nil {
return m.ParentSpanId
}
return ""
}
func (m *Span) GetServiceName() string {
if m != nil {
return m.ServiceName
}
return ""
}
func (m *Span) GetOperationName() string {
if m != nil {
return m.OperationName
}
return ""
}
func (m *Span) GetStartTime() int64 {
if m != nil {
return m.StartTime
}
return 0
}
func (m *Span) GetDuration() int64 {
if m != nil {
return m.Duration
}
return 0
}
func (m *Span) GetLogs() []*Log {
if m != nil {
return m.Logs
}
return nil
}
func (m *Span) GetTags() []*Tag {
if m != nil {
return m.Tags
}
return nil
}
// Log is a timestamped event with a set of tags.
type Log struct {
Timestamp int64 `protobuf:"varint,1,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
Fields []*Tag `protobuf:"bytes,2,rep,name=fields,proto3" json:"fields,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Log) Reset() { *m = Log{} }
func (m *Log) String() string { return proto.CompactTextString(m) }
func (*Log) ProtoMessage() {}
func (*Log) Descriptor() ([]byte, []int) {
return fileDescriptor_fc5f2b88b579999f, []int{1}
}
func (m *Log) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Log.Unmarshal(m, b)
}
func (m *Log) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Log.Marshal(b, m, deterministic)
}
func (m *Log) XXX_Merge(src proto.Message) {
xxx_messageInfo_Log.Merge(m, src)
}
func (m *Log) XXX_Size() int {
return xxx_messageInfo_Log.Size(m)
}
func (m *Log) XXX_DiscardUnknown() {
xxx_messageInfo_Log.DiscardUnknown(m)
}
var xxx_messageInfo_Log proto.InternalMessageInfo
func (m *Log) GetTimestamp() int64 {
if m != nil {
return m.Timestamp
}
return 0
}
func (m *Log) GetFields() []*Tag {
if m != nil {
return m.Fields
}
return nil
}
// Tag is a strongly typed key/value pair. We use 'oneof' protobuf attribute to represent the possible tagTypes
type Tag struct {
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"`
Type Tag_TagType `protobuf:"varint,2,opt,name=type,proto3,enum=Tag_TagType" json:"type,omitempty"`
// Types that are valid to be assigned to Myvalue:
// *Tag_VStr
// *Tag_VLong
// *Tag_VDouble
// *Tag_VBool
// *Tag_VBytes
Myvalue isTag_Myvalue `protobuf_oneof:"myvalue"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Tag) Reset() { *m = Tag{} }
func (m *Tag) String() string { return proto.CompactTextString(m) }
func (*Tag) ProtoMessage() {}
func (*Tag) Descriptor() ([]byte, []int) {
return fileDescriptor_fc5f2b88b579999f, []int{2}
}
func (m *Tag) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Tag.Unmarshal(m, b)
}
func (m *Tag) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Tag.Marshal(b, m, deterministic)
}
func (m *Tag) XXX_Merge(src proto.Message) {
xxx_messageInfo_Tag.Merge(m, src)
}
func (m *Tag) XXX_Size() int {
return xxx_messageInfo_Tag.Size(m)
}
func (m *Tag) XXX_DiscardUnknown() {
xxx_messageInfo_Tag.DiscardUnknown(m)
}
var xxx_messageInfo_Tag proto.InternalMessageInfo
func (m *Tag) GetKey() string {
if m != nil {
return m.Key
}
return ""
}
func (m *Tag) GetType() Tag_TagType {
if m != nil {
return m.Type
}
return Tag_STRING
}
type isTag_Myvalue interface {
isTag_Myvalue()
}
type Tag_VStr struct {
VStr string `protobuf:"bytes,3,opt,name=vStr,proto3,oneof"`
}
type Tag_VLong struct {
VLong int64 `protobuf:"varint,4,opt,name=vLong,proto3,oneof"`
}
type Tag_VDouble struct {
VDouble float64 `protobuf:"fixed64,5,opt,name=vDouble,proto3,oneof"`
}
type Tag_VBool struct {
VBool bool `protobuf:"varint,6,opt,name=vBool,proto3,oneof"`
}
type Tag_VBytes struct {
VBytes []byte `protobuf:"bytes,7,opt,name=vBytes,proto3,oneof"`
}
func (*Tag_VStr) isTag_Myvalue() {}
func (*Tag_VLong) isTag_Myvalue() {}
func (*Tag_VDouble) isTag_Myvalue() {}
func (*Tag_VBool) isTag_Myvalue() {}
func (*Tag_VBytes) isTag_Myvalue() {}
func (m *Tag) GetMyvalue() isTag_Myvalue {
if m != nil {
return m.Myvalue
}
return nil
}
func (m *Tag) GetVStr() string {
if x, ok := m.GetMyvalue().(*Tag_VStr); ok {
return x.VStr
}
return ""
}
func (m *Tag) GetVLong() int64 {
if x, ok := m.GetMyvalue().(*Tag_VLong); ok {
return x.VLong
}
return 0
}
func (m *Tag) GetVDouble() float64 {
if x, ok := m.GetMyvalue().(*Tag_VDouble); ok {
return x.VDouble
}
return 0
}
func (m *Tag) GetVBool() bool {
if x, ok := m.GetMyvalue().(*Tag_VBool); ok {
return x.VBool
}
return false
}
func (m *Tag) GetVBytes() []byte {
if x, ok := m.GetMyvalue().(*Tag_VBytes); ok {
return x.VBytes
}
return nil
}
// XXX_OneofWrappers is for the internal use of the proto package.
func (*Tag) XXX_OneofWrappers() []interface{} {
return []interface{}{
(*Tag_VStr)(nil),
(*Tag_VLong)(nil),
(*Tag_VDouble)(nil),
(*Tag_VBool)(nil),
(*Tag_VBytes)(nil),
}
}
// You can optionally use Batch to send a collection of spans. Spans may not necessarily belong to one traceId.
type Batch struct {
Spans []*Span `protobuf:"bytes,1,rep,name=spans,proto3" json:"spans,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Batch) Reset() { *m = Batch{} }
func (m *Batch) String() string { return proto.CompactTextString(m) }
func (*Batch) ProtoMessage() {}
func (*Batch) Descriptor() ([]byte, []int) {
return fileDescriptor_fc5f2b88b579999f, []int{3}
}
func (m *Batch) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Batch.Unmarshal(m, b)
}
func (m *Batch) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Batch.Marshal(b, m, deterministic)
}
func (m *Batch) XXX_Merge(src proto.Message) {
xxx_messageInfo_Batch.Merge(m, src)
}
func (m *Batch) XXX_Size() int {
return xxx_messageInfo_Batch.Size(m)
}
func (m *Batch) XXX_DiscardUnknown() {
xxx_messageInfo_Batch.DiscardUnknown(m)
}
var xxx_messageInfo_Batch proto.InternalMessageInfo
func (m *Batch) GetSpans() []*Span {
if m != nil {
return m.Spans
}
return nil
}
func init() {
proto.RegisterEnum("Tag_TagType", Tag_TagType_name, Tag_TagType_value)
proto.RegisterType((*Span)(nil), "Span")
proto.RegisterType((*Log)(nil), "Log")
proto.RegisterType((*Tag)(nil), "Tag")
proto.RegisterType((*Batch)(nil), "Batch")
}
func init() { proto.RegisterFile("span.proto", fileDescriptor_fc5f2b88b579999f) }
var fileDescriptor_fc5f2b88b579999f = []byte{
// 456 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x54, 0x92, 0xcd, 0x8a, 0x9c, 0x40,
0x14, 0x85, 0xdb, 0xf6, 0xb7, 0xef, 0x74, 0x82, 0x14, 0x61, 0x28, 0x26, 0xb3, 0x10, 0x19, 0x42,
0xaf, 0x5c, 0x4c, 0x9e, 0xa0, 0x65, 0xc2, 0x74, 0x83, 0x74, 0x87, 0x6a, 0xb3, 0x48, 0x76, 0x35,
0x5a, 0x71, 0x64, 0xd4, 0x2a, 0xac, 0x6a, 0x89, 0xeb, 0xbc, 0x41, 0x9e, 0x38, 0x54, 0xe9, 0xfc,
0x2d, 0x84, 0x7b, 0xbe, 0x73, 0xaf, 0xca, 0xe1, 0x00, 0x48, 0x41, 0xbb, 0x44, 0xf4, 0x5c, 0xf1,
0xf8, 0xdf, 0x12, 0x9c, 0x93, 0xa0, 0x1d, 0xc2, 0xe0, 0xab, 0x9e, 0x16, 0x6c, 0x5f, 0x62, 0x2b,
0xb2, 0x36, 0x2b, 0xf2, 0x2c, 0xd1, 0x25, 0x78, 0xfa, 0x60, 0x5f, 0xe2, 0xa5, 0x31, 0x66, 0x85,
0x62, 0x58, 0x0b, 0xda, 0xb3, 0x4e, 0x9d, 0x26, 0xd7, 0x36, 0xee, 0x3b, 0x86, 0x22, 0xb8, 0x90,
0xac, 0x1f, 0xea, 0x82, 0x1d, 0x68, 0xcb, 0xb0, 0x63, 0x56, 0xde, 0x22, 0x74, 0x03, 0x1f, 0xb8,
0x60, 0x3d, 0x55, 0x35, 0xef, 0xcc, 0x8e, 0x6b, 0x76, 0xde, 0x43, 0x74, 0x0d, 0x2b, 0xa9, 0x68,
0xaf, 0xf2, 0xba, 0x65, 0xd8, 0x8b, 0xac, 0x8d, 0x4d, 0x5e, 0x01, 0xba, 0x82, 0xa0, 0x3c, 0x4f,
0xdb, 0xd8, 0x37, 0xe6, 0x8b, 0x46, 0x18, 0x9c, 0x86, 0x57, 0x12, 0x07, 0x91, 0xbd, 0xb9, 0xb8,
0x75, 0x92, 0x8c, 0x57, 0xc4, 0x10, 0xed, 0x28, 0x5a, 0x49, 0xbc, 0x9a, 0x9d, 0x9c, 0x56, 0xc4,
0x90, 0x78, 0x0b, 0x76, 0xc6, 0x2b, 0xfd, 0x51, 0x55, 0xb7, 0x4c, 0x2a, 0xda, 0x0a, 0x13, 0x8a,
0x4d, 0x5e, 0x01, 0xba, 0x06, 0xef, 0x77, 0xcd, 0x9a, 0x52, 0xe2, 0xe5, 0x9b, 0x17, 0xcc, 0x2c,
0xfe, 0xbb, 0x04, 0x3b, 0xa7, 0x15, 0x0a, 0xc1, 0x7e, 0x62, 0xe3, 0x1c, 0xa9, 0x1e, 0x51, 0x04,
0x8e, 0x1a, 0x05, 0x33, 0x61, 0x7e, 0xbc, 0x5d, 0xeb, 0x2b, 0xfd, 0xe4, 0xa3, 0x60, 0xc4, 0x38,
0xe8, 0x13, 0x38, 0xc3, 0x49, 0xf5, 0x53, 0xa0, 0xbb, 0x05, 0x31, 0x0a, 0x5d, 0x82, 0x3b, 0x64,
0xbc, 0xab, 0x4c, 0x88, 0xf6, 0x6e, 0x41, 0x26, 0x89, 0xae, 0xc0, 0x1f, 0xee, 0xf8, 0xf9, 0xa1,
0x99, 0xa2, 0xb3, 0x76, 0x0b, 0xf2, 0x0c, 0xcc, 0x4d, 0xca, 0x79, 0x63, 0x22, 0x0b, 0xcc, 0x8d,
0x96, 0x08, 0x83, 0x37, 0xa4, 0xa3, 0x62, 0xd2, 0xc4, 0xb5, 0xde, 0x2d, 0xc8, 0xac, 0xe3, 0x2d,
0xf8, 0xf3, 0xcf, 0x20, 0x00, 0xef, 0x94, 0x93, 0xfd, 0xe1, 0x3e, 0x5c, 0xe8, 0xf9, 0xee, 0xf8,
0x23, 0xcd, 0xbe, 0x85, 0x16, 0x0a, 0xc0, 0x49, 0x8f, 0xc7, 0x2c, 0x5c, 0xea, 0x29, 0x3b, 0x1e,
0xee, 0x43, 0x5b, 0xfb, 0xe9, 0xfe, 0xb0, 0x25, 0x3f, 0x43, 0x27, 0x5d, 0x81, 0xdf, 0x8e, 0x03,
0x6d, 0xce, 0x2c, 0xbe, 0x01, 0x37, 0xa5, 0xaa, 0x78, 0x44, 0x9f, 0xc1, 0xd5, 0xad, 0x91, 0xd8,
0x32, 0x59, 0xb9, 0x89, 0xee, 0x07, 0x99, 0x58, 0xfa, 0x05, 0x70, 0xc1, 0xdb, 0x84, 0xfd, 0x11,
0xac, 0xac, 0x69, 0xc2, 0x05, 0xeb, 0x12, 0x5d, 0xbe, 0xba, 0xab, 0xbe, 0x5b, 0xbf, 0x82, 0x47,
0x3a, 0x4a, 0x45, 0x8b, 0xa7, 0x07, 0xcf, 0x54, 0xf6, 0xeb, 0xff, 0x00, 0x00, 0x00, 0xff, 0xff,
0xdd, 0x60, 0xee, 0x6b, 0xc0, 0x02, 0x00, 0x00,
}

View file

@ -0,0 +1,197 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: spanAgent.proto
package haystack
import (
context "context"
fmt "fmt"
proto "github.com/golang/protobuf/proto"
grpc "google.golang.org/grpc"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
type DispatchResult_ResultCode int32
const (
DispatchResult_SUCCESS DispatchResult_ResultCode = 0
DispatchResult_UNKNOWN_ERROR DispatchResult_ResultCode = 1
DispatchResult_RATE_LIMIT_ERROR DispatchResult_ResultCode = 2
)
var DispatchResult_ResultCode_name = map[int32]string{
0: "SUCCESS",
1: "UNKNOWN_ERROR",
2: "RATE_LIMIT_ERROR",
}
var DispatchResult_ResultCode_value = map[string]int32{
"SUCCESS": 0,
"UNKNOWN_ERROR": 1,
"RATE_LIMIT_ERROR": 2,
}
func (x DispatchResult_ResultCode) String() string {
return proto.EnumName(DispatchResult_ResultCode_name, int32(x))
}
func (DispatchResult_ResultCode) EnumDescriptor() ([]byte, []int) {
return fileDescriptor_5a4cb81dd7dcc459, []int{0, 0}
}
type DispatchResult struct {
Code DispatchResult_ResultCode `protobuf:"varint,1,opt,name=code,proto3,enum=DispatchResult_ResultCode" json:"code,omitempty"`
ErrorMessage string `protobuf:"bytes,2,opt,name=error_message,json=errorMessage,proto3" json:"error_message,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *DispatchResult) Reset() { *m = DispatchResult{} }
func (m *DispatchResult) String() string { return proto.CompactTextString(m) }
func (*DispatchResult) ProtoMessage() {}
func (*DispatchResult) Descriptor() ([]byte, []int) {
return fileDescriptor_5a4cb81dd7dcc459, []int{0}
}
func (m *DispatchResult) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_DispatchResult.Unmarshal(m, b)
}
func (m *DispatchResult) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_DispatchResult.Marshal(b, m, deterministic)
}
func (m *DispatchResult) XXX_Merge(src proto.Message) {
xxx_messageInfo_DispatchResult.Merge(m, src)
}
func (m *DispatchResult) XXX_Size() int {
return xxx_messageInfo_DispatchResult.Size(m)
}
func (m *DispatchResult) XXX_DiscardUnknown() {
xxx_messageInfo_DispatchResult.DiscardUnknown(m)
}
var xxx_messageInfo_DispatchResult proto.InternalMessageInfo
func (m *DispatchResult) GetCode() DispatchResult_ResultCode {
if m != nil {
return m.Code
}
return DispatchResult_SUCCESS
}
func (m *DispatchResult) GetErrorMessage() string {
if m != nil {
return m.ErrorMessage
}
return ""
}
func init() {
proto.RegisterEnum("DispatchResult_ResultCode", DispatchResult_ResultCode_name, DispatchResult_ResultCode_value)
proto.RegisterType((*DispatchResult)(nil), "DispatchResult")
}
func init() { proto.RegisterFile("spanAgent.proto", fileDescriptor_5a4cb81dd7dcc459) }
var fileDescriptor_5a4cb81dd7dcc459 = []byte{
// 254 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x5c, 0x90, 0xc1, 0x4a, 0xf3, 0x40,
0x10, 0xc7, 0xbf, 0x2d, 0x9f, 0xda, 0x8e, 0xb6, 0x8d, 0x8b, 0x87, 0x92, 0x53, 0x89, 0x97, 0x9e,
0x56, 0xa9, 0x4f, 0xd0, 0xc6, 0x1c, 0x8a, 0x36, 0x95, 0x4d, 0x8b, 0xe0, 0x25, 0x8c, 0x9b, 0x21,
0x0d, 0xda, 0xec, 0xb2, 0xbb, 0x82, 0x3e, 0x92, 0x6f, 0x29, 0x4d, 0x2a, 0xa2, 0xa7, 0x19, 0x7e,
0x33, 0x03, 0xff, 0xf9, 0xc1, 0xd0, 0x19, 0xac, 0x67, 0x25, 0xd5, 0x5e, 0x18, 0xab, 0xbd, 0x0e,
0x61, 0x0f, 0xda, 0x3e, 0xfa, 0x64, 0x30, 0xb8, 0xad, 0x9c, 0x41, 0xaf, 0xb6, 0x92, 0xdc, 0xdb,
0xab, 0xe7, 0x02, 0xfe, 0x2b, 0x5d, 0xd0, 0x88, 0x8d, 0xd9, 0x64, 0x30, 0x0d, 0xc5, 0xef, 0xb1,
0x68, 0x4b, 0xac, 0x0b, 0x92, 0xcd, 0x1e, 0xbf, 0x84, 0x3e, 0x59, 0xab, 0x6d, 0xbe, 0x23, 0xe7,
0xb0, 0xa4, 0x51, 0x67, 0xcc, 0x26, 0x3d, 0x79, 0xd6, 0xc0, 0x65, 0xcb, 0xa2, 0x39, 0xc0, 0xcf,
0x21, 0x3f, 0x85, 0x93, 0x6c, 0x13, 0xc7, 0x49, 0x96, 0x05, 0xff, 0xf8, 0x39, 0xf4, 0x37, 0xe9,
0x5d, 0xba, 0x7a, 0x4c, 0xf3, 0x44, 0xca, 0x95, 0x0c, 0x18, 0xbf, 0x80, 0x40, 0xce, 0xd6, 0x49,
0x7e, 0xbf, 0x58, 0x2e, 0xd6, 0x07, 0xda, 0x99, 0x5e, 0x41, 0x2f, 0xfb, 0x7e, 0x85, 0x47, 0xd0,
0x2d, 0x0e, 0xc1, 0xf8, 0x91, 0xd8, 0xf3, 0x70, 0xf8, 0x27, 0xea, 0xfc, 0x1a, 0x22, 0xa5, 0x77,
0x82, 0xde, 0x0d, 0x15, 0x15, 0x0a, 0x6d, 0xa8, 0x16, 0xde, 0xa2, 0xaa, 0xea, 0x52, 0x60, 0x23,
0x04, 0x4d, 0xf5, 0xc0, 0x9e, 0xba, 0x5b, 0xfc, 0x70, 0x1e, 0xd5, 0xcb, 0xf3, 0x71, 0x63, 0xe5,
0xe6, 0x2b, 0x00, 0x00, 0xff, 0xff, 0x30, 0xb9, 0x7d, 0x07, 0x34, 0x01, 0x00, 0x00,
}
// Reference imports to suppress errors if they are not otherwise used.
var _ context.Context
var _ grpc.ClientConn
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
const _ = grpc.SupportPackageIsVersion4
// SpanAgentClient is the client API for SpanAgent service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.
type SpanAgentClient interface {
Dispatch(ctx context.Context, in *Span, opts ...grpc.CallOption) (*DispatchResult, error)
}
type spanAgentClient struct {
cc *grpc.ClientConn
}
func NewSpanAgentClient(cc *grpc.ClientConn) SpanAgentClient {
return &spanAgentClient{cc}
}
func (c *spanAgentClient) Dispatch(ctx context.Context, in *Span, opts ...grpc.CallOption) (*DispatchResult, error) {
out := new(DispatchResult)
err := c.cc.Invoke(ctx, "/SpanAgent/dispatch", in, out, opts...)
if err != nil {
return nil, err
}
return out, nil
}
// SpanAgentServer is the server API for SpanAgent service.
type SpanAgentServer interface {
Dispatch(context.Context, *Span) (*DispatchResult, error)
}
func RegisterSpanAgentServer(s *grpc.Server, srv SpanAgentServer) {
s.RegisterService(&_SpanAgent_serviceDesc, srv)
}
func _SpanAgent_Dispatch_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(Span)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(SpanAgentServer).Dispatch(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: "/SpanAgent/Dispatch",
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(SpanAgentServer).Dispatch(ctx, req.(*Span))
}
return interceptor(ctx, in, info, handler)
}
var _SpanAgent_serviceDesc = grpc.ServiceDesc{
ServiceName: "SpanAgent",
HandlerType: (*SpanAgentServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "dispatch",
Handler: _SpanAgent_Dispatch_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "spanAgent.proto",
}

View file

@ -0,0 +1,80 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"fmt"
)
/*SpanContext implements opentracing.spanContext*/
type SpanContext struct {
// traceID represents globally unique ID of the trace.
TraceID string
// spanID represents span ID that must be unique within its trace
SpanID string
// parentID refers to the ID of the parent span.
// Should be empty if the current span is a root span.
ParentID string
//Context baggage. The is a snapshot in time.
Baggage map[string]string
// set to true if extracted using a extractor in tracer
IsExtractedContext bool
}
// IsValid indicates whether this context actually represents a valid trace.
func (context SpanContext) IsValid() bool {
return context.TraceID != "" && context.SpanID != ""
}
/*ForeachBaggageItem implements opentracing.spancontext*/
func (context SpanContext) ForeachBaggageItem(handler func(k, v string) bool) {
for k, v := range context.Baggage {
if !handler(k, v) {
break
}
}
}
// WithBaggageItem creates a new context with an extra baggage item.
func (context SpanContext) WithBaggageItem(key, value string) *SpanContext {
var newBaggage map[string]string
if context.Baggage == nil {
newBaggage = map[string]string{key: value}
} else {
newBaggage = make(map[string]string, len(context.Baggage)+1)
for k, v := range context.Baggage {
newBaggage[k] = v
}
newBaggage[key] = value
}
return &SpanContext{
TraceID: context.TraceID,
SpanID: context.SpanID,
ParentID: context.ParentID,
Baggage: newBaggage,
}
}
/*ToString represents the string*/
func (context SpanContext) ToString() string {
return fmt.Sprintf("%+v", context)
}

View file

@ -0,0 +1,214 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"io"
"time"
"github.com/google/uuid"
"github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
)
/*Tracer implements the opentracing.tracer*/
type Tracer struct {
serviceName string
logger Logger
dispatcher Dispatcher
commonTags []opentracing.Tag
timeNow func() time.Time
idGenerator func() string
propagators map[interface{}]Propagator
useDualSpanMode bool
}
/*NewTracer creates a new tracer*/
func NewTracer(
serviceName string,
dispatcher Dispatcher,
options ...TracerOption,
) (opentracing.Tracer, io.Closer) {
tracer := &Tracer{
serviceName: serviceName,
dispatcher: dispatcher,
useDualSpanMode: false,
}
tracer.propagators = make(map[interface{}]Propagator)
tracer.propagators[opentracing.TextMap] = NewDefaultTextMapPropagator()
tracer.propagators[opentracing.HTTPHeaders] = NewTextMapPropagator(PropagatorOpts{}, URLCodex{})
for _, option := range options {
option(tracer)
}
if tracer.timeNow == nil {
tracer.timeNow = time.Now
}
if tracer.logger == nil {
tracer.logger = NullLogger{}
}
if tracer.idGenerator == nil {
tracer.idGenerator = func() string {
_uuid, err := uuid.NewUUID()
if err != nil {
panic(err)
}
return _uuid.String()
}
}
dispatcher.SetLogger(tracer.logger)
return tracer, tracer
}
/*StartSpan starts a new span*/
func (tracer *Tracer) StartSpan(
operationName string,
options ...opentracing.StartSpanOption,
) opentracing.Span {
sso := opentracing.StartSpanOptions{}
for _, o := range options {
o.Apply(&sso)
}
if sso.StartTime.IsZero() {
sso.StartTime = tracer.timeNow()
}
var followsFromIsParent = false
var parent *SpanContext
for _, ref := range sso.References {
if ref.Type == opentracing.ChildOfRef {
if parent == nil || followsFromIsParent {
parent = ref.ReferencedContext.(*SpanContext)
}
} else if ref.Type == opentracing.FollowsFromRef {
if parent == nil {
parent = ref.ReferencedContext.(*SpanContext)
followsFromIsParent = true
}
}
}
spanContext := tracer.createSpanContext(parent, tracer.isServerSpan(sso.Tags))
span := &_Span{
tracer: tracer,
context: spanContext,
operationName: operationName,
startTime: sso.StartTime,
duration: 0,
}
for _, tag := range tracer.Tags() {
span.SetTag(tag.Key, tag.Value)
}
for k, v := range sso.Tags {
span.SetTag(k, v)
}
return span
}
func (tracer *Tracer) isServerSpan(spanTags map[string]interface{}) bool {
if spanKind, ok := spanTags[string(ext.SpanKind)]; ok && spanKind == "server" {
return true
}
return false
}
func (tracer *Tracer) createSpanContext(parent *SpanContext, isServerSpan bool) *SpanContext {
if parent == nil || !parent.IsValid() {
return &SpanContext{
TraceID: tracer.idGenerator(),
SpanID: tracer.idGenerator(),
}
}
// This is a check to see if the tracer is configured to support single
// single span type (Zipkin style shared span id) or
// dual span type (client and server having their own span ids ).
// a. If tracer is not of dualSpanType and if it is a server span then we
// just return the parent context with the same shared span ids
// b. If tracer is not of dualSpanType and if the parent context is an extracted one from the wire
// then we assume this is the first span in the server and so just return the parent context
// with the same shared span ids
if !tracer.useDualSpanMode && (isServerSpan || parent.IsExtractedContext) {
return &SpanContext{
TraceID: parent.TraceID,
SpanID: parent.SpanID,
ParentID: parent.ParentID,
Baggage: parent.Baggage,
IsExtractedContext: false,
}
}
return &SpanContext{
TraceID: parent.TraceID,
SpanID: tracer.idGenerator(),
ParentID: parent.SpanID,
Baggage: parent.Baggage,
IsExtractedContext: false,
}
}
/*Inject implements Inject() method of opentracing.Tracer*/
func (tracer *Tracer) Inject(ctx opentracing.SpanContext, format interface{}, carrier interface{}) error {
c, ok := ctx.(*SpanContext)
if !ok {
return opentracing.ErrInvalidSpanContext
}
if injector, ok := tracer.propagators[format]; ok {
return injector.Inject(c, carrier)
}
return opentracing.ErrUnsupportedFormat
}
/*Extract implements Extract() method of opentracing.Tracer*/
func (tracer *Tracer) Extract(
format interface{},
carrier interface{},
) (opentracing.SpanContext, error) {
if extractor, ok := tracer.propagators[format]; ok {
return extractor.Extract(carrier)
}
return nil, opentracing.ErrUnsupportedFormat
}
/*Tags return all common tags */
func (tracer *Tracer) Tags() []opentracing.Tag {
return tracer.commonTags
}
/*DispatchSpan dispatches the span to a dispatcher*/
func (tracer *Tracer) DispatchSpan(span *_Span) {
if tracer.dispatcher != nil {
tracer.dispatcher.Dispatch(span)
}
}
/*Close closes the tracer*/
func (tracer *Tracer) Close() error {
if tracer.dispatcher != nil {
tracer.dispatcher.Close()
}
return nil
}

View file

@ -0,0 +1,59 @@
/*
* Copyright 2018 Expedia Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package haystack
import (
"github.com/opentracing/opentracing-go"
)
// TracerOption is a function that sets some option on the tracer
type TracerOption func(tracer *Tracer)
/*TracerOptions a list of tracer options*/
type TracerOptions struct{}
/*TracerOptionsFactory factory to create multiple tracer options*/
var TracerOptionsFactory TracerOptions
/*Propagator registers a new Propagator*/
func (t TracerOptions) Propagator(format interface{}, propagator Propagator) TracerOption {
return func(tracer *Tracer) {
tracer.propagators[format] = propagator
}
}
/*Tag adds a common tag in every span*/
func (t TracerOptions) Tag(key string, value interface{}) TracerOption {
return func(tracer *Tracer) {
tracer.commonTags = append(tracer.commonTags, opentracing.Tag{Key: key, Value: value})
}
}
/*Logger set the logger type*/
func (t TracerOptions) Logger(logger Logger) TracerOption {
return func(tracer *Tracer) {
tracer.logger = logger
}
}
/*UseDualSpanMode sets the tracer in dual span mode*/
func (t TracerOptions) UseDualSpanMode() TracerOption {
return func(tracer *Tracer) {
tracer.useDualSpanMode = true
}
}

View file

@ -1,7 +1,4 @@
Go support for Protocol Buffers - Google's data interchange format
Copyright 2010 The Go Authors. All rights reserved.
https://github.com/golang/protobuf
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are

View file

@ -186,7 +186,6 @@ func (p *Buffer) DecodeVarint() (x uint64, err error) {
if b&0x80 == 0 {
goto done
}
// x -= 0x80 << 63 // Always zero.
return 0, errOverflow

63
vendor/github.com/golang/protobuf/proto/deprecated.go generated vendored Normal file
View file

@ -0,0 +1,63 @@
// Go support for Protocol Buffers - Google's data interchange format
//
// Copyright 2018 The Go Authors. All rights reserved.
// https://github.com/golang/protobuf
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
package proto
import "errors"
// Deprecated: do not use.
type Stats struct{ Emalloc, Dmalloc, Encode, Decode, Chit, Cmiss, Size uint64 }
// Deprecated: do not use.
func GetStats() Stats { return Stats{} }
// Deprecated: do not use.
func MarshalMessageSet(interface{}) ([]byte, error) {
return nil, errors.New("proto: not implemented")
}
// Deprecated: do not use.
func UnmarshalMessageSet([]byte, interface{}) error {
return errors.New("proto: not implemented")
}
// Deprecated: do not use.
func MarshalMessageSetJSON(interface{}) ([]byte, error) {
return nil, errors.New("proto: not implemented")
}
// Deprecated: do not use.
func UnmarshalMessageSetJSON([]byte, interface{}) error {
return errors.New("proto: not implemented")
}
// Deprecated: do not use.
func RegisterMessageSetType(Message, int32, string) {}

View file

@ -37,27 +37,9 @@ package proto
import (
"errors"
"fmt"
"reflect"
)
// RequiredNotSetError is the error returned if Marshal is called with
// a protocol buffer struct whose required fields have not
// all been initialized. It is also the error returned if Unmarshal is
// called with an encoded protocol buffer that does not include all the
// required fields.
//
// When printed, RequiredNotSetError reports the first unset required field in a
// message. If the field cannot be precisely determined, it is reported as
// "{Unknown}".
type RequiredNotSetError struct {
field string
}
func (e *RequiredNotSetError) Error() string {
return fmt.Sprintf("proto: required field %q not set", e.field)
}
var (
// errRepeatedHasNil is the error returned if Marshal is called with
// a struct with a repeated field containing a nil element.

View file

@ -246,7 +246,8 @@ func equalExtMap(base reflect.Type, em1, em2 map[int32]Extension) bool {
return false
}
m1, m2 := e1.value, e2.value
m1 := extensionAsLegacyType(e1.value)
m2 := extensionAsLegacyType(e2.value)
if m1 == nil && m2 == nil {
// Both have only encoded form.

View file

@ -185,9 +185,25 @@ type Extension struct {
// extension will have only enc set. When such an extension is
// accessed using GetExtension (or GetExtensions) desc and value
// will be set.
desc *ExtensionDesc
desc *ExtensionDesc
// value is a concrete value for the extension field. Let the type of
// desc.ExtensionType be the "API type" and the type of Extension.value
// be the "storage type". The API type and storage type are the same except:
// * For scalars (except []byte), the API type uses *T,
// while the storage type uses T.
// * For repeated fields, the API type uses []T, while the storage type
// uses *[]T.
//
// The reason for the divergence is so that the storage type more naturally
// matches what is expected of when retrieving the values through the
// protobuf reflection APIs.
//
// The value may only be populated if desc is also populated.
value interface{}
enc []byte
// enc is the raw bytes for the extension field.
enc []byte
}
// SetRawExtension is for testing only.
@ -334,7 +350,7 @@ func GetExtension(pb Message, extension *ExtensionDesc) (interface{}, error) {
// descriptors with the same field number.
return nil, errors.New("proto: descriptor conflict")
}
return e.value, nil
return extensionAsLegacyType(e.value), nil
}
if extension.ExtensionType == nil {
@ -349,11 +365,11 @@ func GetExtension(pb Message, extension *ExtensionDesc) (interface{}, error) {
// Remember the decoded version and drop the encoded version.
// That way it is safe to mutate what we return.
e.value = v
e.value = extensionAsStorageType(v)
e.desc = extension
e.enc = nil
emap[extension.Field] = e
return e.value, nil
return extensionAsLegacyType(e.value), nil
}
// defaultExtensionValue returns the default value for extension.
@ -488,7 +504,7 @@ func SetExtension(pb Message, extension *ExtensionDesc, value interface{}) error
}
typ := reflect.TypeOf(extension.ExtensionType)
if typ != reflect.TypeOf(value) {
return errors.New("proto: bad extension value type")
return fmt.Errorf("proto: bad extension value type. got: %T, want: %T", value, extension.ExtensionType)
}
// nil extension values need to be caught early, because the
// encoder can't distinguish an ErrNil due to a nil extension
@ -500,7 +516,7 @@ func SetExtension(pb Message, extension *ExtensionDesc, value interface{}) error
}
extmap := epb.extensionsWrite()
extmap[extension.Field] = Extension{desc: extension, value: value}
extmap[extension.Field] = Extension{desc: extension, value: extensionAsStorageType(value)}
return nil
}
@ -541,3 +557,51 @@ func RegisterExtension(desc *ExtensionDesc) {
func RegisteredExtensions(pb Message) map[int32]*ExtensionDesc {
return extensionMaps[reflect.TypeOf(pb).Elem()]
}
// extensionAsLegacyType converts an value in the storage type as the API type.
// See Extension.value.
func extensionAsLegacyType(v interface{}) interface{} {
switch rv := reflect.ValueOf(v); rv.Kind() {
case reflect.Bool, reflect.Int32, reflect.Int64, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64, reflect.String:
// Represent primitive types as a pointer to the value.
rv2 := reflect.New(rv.Type())
rv2.Elem().Set(rv)
v = rv2.Interface()
case reflect.Ptr:
// Represent slice types as the value itself.
switch rv.Type().Elem().Kind() {
case reflect.Slice:
if rv.IsNil() {
v = reflect.Zero(rv.Type().Elem()).Interface()
} else {
v = rv.Elem().Interface()
}
}
}
return v
}
// extensionAsStorageType converts an value in the API type as the storage type.
// See Extension.value.
func extensionAsStorageType(v interface{}) interface{} {
switch rv := reflect.ValueOf(v); rv.Kind() {
case reflect.Ptr:
// Represent slice types as the value itself.
switch rv.Type().Elem().Kind() {
case reflect.Bool, reflect.Int32, reflect.Int64, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64, reflect.String:
if rv.IsNil() {
v = reflect.Zero(rv.Type().Elem()).Interface()
} else {
v = rv.Elem().Interface()
}
}
case reflect.Slice:
// Represent slice types as a pointer to the value.
if rv.Type().Elem().Kind() != reflect.Uint8 {
rv2 := reflect.New(rv.Type())
rv2.Elem().Set(rv)
v = rv2.Interface()
}
}
return v
}

View file

@ -265,7 +265,6 @@ package proto
import (
"encoding/json"
"errors"
"fmt"
"log"
"reflect"
@ -274,7 +273,66 @@ import (
"sync"
)
var errInvalidUTF8 = errors.New("proto: invalid UTF-8 string")
// RequiredNotSetError is an error type returned by either Marshal or Unmarshal.
// Marshal reports this when a required field is not initialized.
// Unmarshal reports this when a required field is missing from the wire data.
type RequiredNotSetError struct{ field string }
func (e *RequiredNotSetError) Error() string {
if e.field == "" {
return fmt.Sprintf("proto: required field not set")
}
return fmt.Sprintf("proto: required field %q not set", e.field)
}
func (e *RequiredNotSetError) RequiredNotSet() bool {
return true
}
type invalidUTF8Error struct{ field string }
func (e *invalidUTF8Error) Error() string {
if e.field == "" {
return "proto: invalid UTF-8 detected"
}
return fmt.Sprintf("proto: field %q contains invalid UTF-8", e.field)
}
func (e *invalidUTF8Error) InvalidUTF8() bool {
return true
}
// errInvalidUTF8 is a sentinel error to identify fields with invalid UTF-8.
// This error should not be exposed to the external API as such errors should
// be recreated with the field information.
var errInvalidUTF8 = &invalidUTF8Error{}
// isNonFatal reports whether the error is either a RequiredNotSet error
// or a InvalidUTF8 error.
func isNonFatal(err error) bool {
if re, ok := err.(interface{ RequiredNotSet() bool }); ok && re.RequiredNotSet() {
return true
}
if re, ok := err.(interface{ InvalidUTF8() bool }); ok && re.InvalidUTF8() {
return true
}
return false
}
type nonFatal struct{ E error }
// Merge merges err into nf and reports whether it was successful.
// Otherwise it returns false for any fatal non-nil errors.
func (nf *nonFatal) Merge(err error) (ok bool) {
if err == nil {
return true // not an error
}
if !isNonFatal(err) {
return false // fatal error
}
if nf.E == nil {
nf.E = err // store first instance of non-fatal error
}
return true
}
// Message is implemented by generated protocol buffer messages.
type Message interface {
@ -283,26 +341,6 @@ type Message interface {
ProtoMessage()
}
// Stats records allocation details about the protocol buffer encoders
// and decoders. Useful for tuning the library itself.
type Stats struct {
Emalloc uint64 // mallocs in encode
Dmalloc uint64 // mallocs in decode
Encode uint64 // number of encodes
Decode uint64 // number of decodes
Chit uint64 // number of cache hits
Cmiss uint64 // number of cache misses
Size uint64 // number of sizes
}
// Set to true to enable stats collection.
const collectStats = false
var stats Stats
// GetStats returns a copy of the global Stats structure.
func GetStats() Stats { return stats }
// A Buffer is a buffer manager for marshaling and unmarshaling
// protocol buffers. It may be reused between invocations to
// reduce memory usage. It is not necessary to use a Buffer;
@ -902,13 +940,19 @@ func isProto3Zero(v reflect.Value) bool {
return false
}
// ProtoPackageIsVersion2 is referenced from generated protocol buffer files
// to assert that that code is compatible with this version of the proto package.
const ProtoPackageIsVersion2 = true
const (
// ProtoPackageIsVersion3 is referenced from generated protocol buffer files
// to assert that that code is compatible with this version of the proto package.
ProtoPackageIsVersion3 = true
// ProtoPackageIsVersion1 is referenced from generated protocol buffer files
// to assert that that code is compatible with this version of the proto package.
const ProtoPackageIsVersion1 = true
// ProtoPackageIsVersion2 is referenced from generated protocol buffer files
// to assert that that code is compatible with this version of the proto package.
ProtoPackageIsVersion2 = true
// ProtoPackageIsVersion1 is referenced from generated protocol buffer files
// to assert that that code is compatible with this version of the proto package.
ProtoPackageIsVersion1 = true
)
// InternalMessageInfo is a type used internally by generated .pb.go files.
// This type is not intended to be used by non-generated code.

View file

@ -36,13 +36,7 @@ package proto
*/
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"reflect"
"sort"
"sync"
)
// errNoMessageTypeID occurs when a protocol buffer does not have a message type ID.
@ -145,46 +139,9 @@ func skipVarint(buf []byte) []byte {
return buf[i+1:]
}
// MarshalMessageSet encodes the extension map represented by m in the message set wire format.
// It is called by generated Marshal methods on protocol buffer messages with the message_set_wire_format option.
func MarshalMessageSet(exts interface{}) ([]byte, error) {
return marshalMessageSet(exts, false)
}
// marshaMessageSet implements above function, with the opt to turn on / off deterministic during Marshal.
func marshalMessageSet(exts interface{}, deterministic bool) ([]byte, error) {
switch exts := exts.(type) {
case *XXX_InternalExtensions:
var u marshalInfo
siz := u.sizeMessageSet(exts)
b := make([]byte, 0, siz)
return u.appendMessageSet(b, exts, deterministic)
case map[int32]Extension:
// This is an old-style extension map.
// Wrap it in a new-style XXX_InternalExtensions.
ie := XXX_InternalExtensions{
p: &struct {
mu sync.Mutex
extensionMap map[int32]Extension
}{
extensionMap: exts,
},
}
var u marshalInfo
siz := u.sizeMessageSet(&ie)
b := make([]byte, 0, siz)
return u.appendMessageSet(b, &ie, deterministic)
default:
return nil, errors.New("proto: not an extension map")
}
}
// UnmarshalMessageSet decodes the extension map encoded in buf in the message set wire format.
// unmarshalMessageSet decodes the extension map encoded in buf in the message set wire format.
// It is called by Unmarshal methods on protocol buffer messages with the message_set_wire_format option.
func UnmarshalMessageSet(buf []byte, exts interface{}) error {
func unmarshalMessageSet(buf []byte, exts interface{}) error {
var m map[int32]Extension
switch exts := exts.(type) {
case *XXX_InternalExtensions:
@ -222,93 +179,3 @@ func UnmarshalMessageSet(buf []byte, exts interface{}) error {
}
return nil
}
// MarshalMessageSetJSON encodes the extension map represented by m in JSON format.
// It is called by generated MarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
func MarshalMessageSetJSON(exts interface{}) ([]byte, error) {
var m map[int32]Extension
switch exts := exts.(type) {
case *XXX_InternalExtensions:
var mu sync.Locker
m, mu = exts.extensionsRead()
if m != nil {
// Keep the extensions map locked until we're done marshaling to prevent
// races between marshaling and unmarshaling the lazily-{en,de}coded
// values.
mu.Lock()
defer mu.Unlock()
}
case map[int32]Extension:
m = exts
default:
return nil, errors.New("proto: not an extension map")
}
var b bytes.Buffer
b.WriteByte('{')
// Process the map in key order for deterministic output.
ids := make([]int32, 0, len(m))
for id := range m {
ids = append(ids, id)
}
sort.Sort(int32Slice(ids)) // int32Slice defined in text.go
for i, id := range ids {
ext := m[id]
msd, ok := messageSetMap[id]
if !ok {
// Unknown type; we can't render it, so skip it.
continue
}
if i > 0 && b.Len() > 1 {
b.WriteByte(',')
}
fmt.Fprintf(&b, `"[%s]":`, msd.name)
x := ext.value
if x == nil {
x = reflect.New(msd.t.Elem()).Interface()
if err := Unmarshal(ext.enc, x.(Message)); err != nil {
return nil, err
}
}
d, err := json.Marshal(x)
if err != nil {
return nil, err
}
b.Write(d)
}
b.WriteByte('}')
return b.Bytes(), nil
}
// UnmarshalMessageSetJSON decodes the extension map encoded in buf in JSON format.
// It is called by generated UnmarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
func UnmarshalMessageSetJSON(buf []byte, exts interface{}) error {
// Common-case fast path.
if len(buf) == 0 || bytes.Equal(buf, []byte("{}")) {
return nil
}
// This is fairly tricky, and it's not clear that it is needed.
return errors.New("TODO: UnmarshalMessageSetJSON not yet implemented")
}
// A global registry of types that can be used in a MessageSet.
var messageSetMap = make(map[int32]messageSetDesc)
type messageSetDesc struct {
t reflect.Type // pointer to struct
name string
}
// RegisterMessageSetType is called from the generated code.
func RegisterMessageSetType(m Message, fieldNum int32, name string) {
messageSetMap[fieldNum] = messageSetDesc{
t: reflect.TypeOf(m),
name: name,
}
}

View file

@ -79,10 +79,13 @@ func toPointer(i *Message) pointer {
// toAddrPointer converts an interface to a pointer that points to
// the interface data.
func toAddrPointer(i *interface{}, isptr bool) pointer {
func toAddrPointer(i *interface{}, isptr, deref bool) pointer {
v := reflect.ValueOf(*i)
u := reflect.New(v.Type())
u.Elem().Set(v)
if deref {
u = u.Elem()
}
return pointer{v: u}
}

View file

@ -85,16 +85,21 @@ func toPointer(i *Message) pointer {
// toAddrPointer converts an interface to a pointer that points to
// the interface data.
func toAddrPointer(i *interface{}, isptr bool) pointer {
func toAddrPointer(i *interface{}, isptr, deref bool) (p pointer) {
// Super-tricky - read or get the address of data word of interface value.
if isptr {
// The interface is of pointer type, thus it is a direct interface.
// The data word is the pointer data itself. We take its address.
return pointer{p: unsafe.Pointer(uintptr(unsafe.Pointer(i)) + ptrSize)}
p = pointer{p: unsafe.Pointer(uintptr(unsafe.Pointer(i)) + ptrSize)}
} else {
// The interface is not of pointer type. The data word is the pointer
// to the data.
p = pointer{p: (*[2]unsafe.Pointer)(unsafe.Pointer(i))[1]}
}
// The interface is not of pointer type. The data word is the pointer
// to the data.
return pointer{p: (*[2]unsafe.Pointer)(unsafe.Pointer(i))[1]}
if deref {
p.p = *(*unsafe.Pointer)(p.p)
}
return p
}
// valToPointer converts v to a pointer. v must be of pointer type.

View file

@ -139,7 +139,7 @@ type Properties struct {
Repeated bool
Packed bool // relevant for repeated primitives only
Enum string // set for enum types only
proto3 bool // whether this is known to be a proto3 field; set for []byte only
proto3 bool // whether this is known to be a proto3 field
oneof bool // whether this is a oneof field
Default string // default value
@ -148,9 +148,9 @@ type Properties struct {
stype reflect.Type // set for struct types only
sprop *StructProperties // set for struct types only
mtype reflect.Type // set for map types only
mkeyprop *Properties // set for map types only
mvalprop *Properties // set for map types only
mtype reflect.Type // set for map types only
MapKeyProp *Properties // set for map types only
MapValProp *Properties // set for map types only
}
// String formats the properties in the protobuf struct field tag style.
@ -275,16 +275,16 @@ func (p *Properties) setFieldProps(typ reflect.Type, f *reflect.StructField, loc
case reflect.Map:
p.mtype = t1
p.mkeyprop = &Properties{}
p.mkeyprop.init(reflect.PtrTo(p.mtype.Key()), "Key", f.Tag.Get("protobuf_key"), nil, lockGetProp)
p.mvalprop = &Properties{}
p.MapKeyProp = &Properties{}
p.MapKeyProp.init(reflect.PtrTo(p.mtype.Key()), "Key", f.Tag.Get("protobuf_key"), nil, lockGetProp)
p.MapValProp = &Properties{}
vtype := p.mtype.Elem()
if vtype.Kind() != reflect.Ptr && vtype.Kind() != reflect.Slice {
// The value type is not a message (*T) or bytes ([]byte),
// so we need encoders for the pointer to this type.
vtype = reflect.PtrTo(vtype)
}
p.mvalprop.init(vtype, "Value", f.Tag.Get("protobuf_val"), nil, lockGetProp)
p.MapValProp.init(vtype, "Value", f.Tag.Get("protobuf_val"), nil, lockGetProp)
}
if p.stype != nil {
@ -334,9 +334,6 @@ func GetProperties(t reflect.Type) *StructProperties {
sprop, ok := propertiesMap[t]
propertiesMu.RUnlock()
if ok {
if collectStats {
stats.Chit++
}
return sprop
}
@ -346,17 +343,20 @@ func GetProperties(t reflect.Type) *StructProperties {
return sprop
}
type (
oneofFuncsIface interface {
XXX_OneofFuncs() (func(Message, *Buffer) error, func(Message, int, int, *Buffer) (bool, error), func(Message) int, []interface{})
}
oneofWrappersIface interface {
XXX_OneofWrappers() []interface{}
}
)
// getPropertiesLocked requires that propertiesMu is held.
func getPropertiesLocked(t reflect.Type) *StructProperties {
if prop, ok := propertiesMap[t]; ok {
if collectStats {
stats.Chit++
}
return prop
}
if collectStats {
stats.Cmiss++
}
prop := new(StructProperties)
// in case of recursive protos, fill this in now.
@ -391,13 +391,14 @@ func getPropertiesLocked(t reflect.Type) *StructProperties {
// Re-order prop.order.
sort.Sort(prop)
type oneofMessage interface {
XXX_OneofFuncs() (func(Message, *Buffer) error, func(Message, int, int, *Buffer) (bool, error), func(Message) int, []interface{})
var oots []interface{}
switch m := reflect.Zero(reflect.PtrTo(t)).Interface().(type) {
case oneofFuncsIface:
_, _, _, oots = m.XXX_OneofFuncs()
case oneofWrappersIface:
oots = m.XXX_OneofWrappers()
}
if om, ok := reflect.Zero(reflect.PtrTo(t)).Interface().(oneofMessage); ok {
var oots []interface{}
_, _, _, oots = om.XXX_OneofFuncs()
if len(oots) > 0 {
// Interpret oneof metadata.
prop.OneofTypes = make(map[string]*OneofProperties)
for _, oot := range oots {

View file

@ -87,6 +87,7 @@ type marshalElemInfo struct {
sizer sizer
marshaler marshaler
isptr bool // elem is pointer typed, thus interface of this type is a direct interface (extension only)
deref bool // dereference the pointer before operating on it; implies isptr
}
var (
@ -231,7 +232,7 @@ func (u *marshalInfo) marshal(b []byte, ptr pointer, deterministic bool) ([]byte
return b, err
}
var err, errreq error
var err, errLater error
// The old marshaler encodes extensions at beginning.
if u.extensions.IsValid() {
e := ptr.offset(u.extensions).toExtensions()
@ -252,11 +253,13 @@ func (u *marshalInfo) marshal(b []byte, ptr pointer, deterministic bool) ([]byte
}
}
for _, f := range u.fields {
if f.required && errreq == nil {
if f.required {
if ptr.offset(f.field).getPointer().isNil() {
// Required field is not set.
// We record the error but keep going, to give a complete marshaling.
errreq = &RequiredNotSetError{f.name}
if errLater == nil {
errLater = &RequiredNotSetError{f.name}
}
continue
}
}
@ -269,14 +272,21 @@ func (u *marshalInfo) marshal(b []byte, ptr pointer, deterministic bool) ([]byte
if err1, ok := err.(*RequiredNotSetError); ok {
// Required field in submessage is not set.
// We record the error but keep going, to give a complete marshaling.
if errreq == nil {
errreq = &RequiredNotSetError{f.name + "." + err1.field}
if errLater == nil {
errLater = &RequiredNotSetError{f.name + "." + err1.field}
}
continue
}
if err == errRepeatedHasNil {
err = errors.New("proto: repeated field " + f.name + " has nil element")
}
if err == errInvalidUTF8 {
if errLater == nil {
fullName := revProtoTypes[reflect.PtrTo(u.typ)] + "." + f.name
errLater = &invalidUTF8Error{fullName}
}
continue
}
return b, err
}
}
@ -284,7 +294,7 @@ func (u *marshalInfo) marshal(b []byte, ptr pointer, deterministic bool) ([]byte
s := *ptr.offset(u.unrecognized).toBytes()
b = append(b, s...)
}
return b, errreq
return b, errLater
}
// computeMarshalInfo initializes the marshal info.
@ -311,8 +321,11 @@ func (u *marshalInfo) computeMarshalInfo() {
// get oneof implementers
var oneofImplementers []interface{}
if m, ok := reflect.Zero(reflect.PtrTo(t)).Interface().(oneofMessage); ok {
switch m := reflect.Zero(reflect.PtrTo(t)).Interface().(type) {
case oneofFuncsIface:
_, _, _, oneofImplementers = m.XXX_OneofFuncs()
case oneofWrappersIface:
oneofImplementers = m.XXX_OneofWrappers()
}
n := t.NumField()
@ -398,13 +411,22 @@ func (u *marshalInfo) getExtElemInfo(desc *ExtensionDesc) *marshalElemInfo {
panic("tag is not an integer")
}
wt := wiretype(tags[0])
if t.Kind() == reflect.Ptr && t.Elem().Kind() != reflect.Struct {
t = t.Elem()
}
sizer, marshaler := typeMarshaler(t, tags, false, false)
var deref bool
if t.Kind() == reflect.Slice && t.Elem().Kind() != reflect.Uint8 {
t = reflect.PtrTo(t)
deref = true
}
e = &marshalElemInfo{
wiretag: uint64(tag)<<3 | wt,
tagsize: SizeVarint(uint64(tag) << 3),
sizer: sizer,
marshaler: marshaler,
isptr: t.Kind() == reflect.Ptr,
deref: deref,
}
// update cache
@ -439,7 +461,7 @@ func (fi *marshalFieldInfo) computeMarshalFieldInfo(f *reflect.StructField) {
func (fi *marshalFieldInfo) computeOneofFieldInfo(f *reflect.StructField, oneofImplementers []interface{}) {
fi.field = toField(f)
fi.wiretag = 1<<31 - 1 // Use a large tag number, make oneofs sorted at the end. This tag will not appear on the wire.
fi.wiretag = math.MaxInt32 // Use a large tag number, make oneofs sorted at the end. This tag will not appear on the wire.
fi.isPointer = true
fi.sizer, fi.marshaler = makeOneOfMarshaler(fi, f)
fi.oneofElems = make(map[reflect.Type]*marshalElemInfo)
@ -467,10 +489,6 @@ func (fi *marshalFieldInfo) computeOneofFieldInfo(f *reflect.StructField, oneofI
}
}
type oneofMessage interface {
XXX_OneofFuncs() (func(Message, *Buffer) error, func(Message, int, int, *Buffer) (bool, error), func(Message) int, []interface{})
}
// wiretype returns the wire encoding of the type.
func wiretype(encoding string) uint64 {
switch encoding {
@ -530,6 +548,7 @@ func typeMarshaler(t reflect.Type, tags []string, nozero, oneof bool) (sizer, ma
packed := false
proto3 := false
validateUTF8 := true
for i := 2; i < len(tags); i++ {
if tags[i] == "packed" {
packed = true
@ -538,6 +557,7 @@ func typeMarshaler(t reflect.Type, tags []string, nozero, oneof bool) (sizer, ma
proto3 = true
}
}
validateUTF8 = validateUTF8 && proto3
switch t.Kind() {
case reflect.Bool:
@ -735,6 +755,18 @@ func typeMarshaler(t reflect.Type, tags []string, nozero, oneof bool) (sizer, ma
}
return sizeFloat64Value, appendFloat64Value
case reflect.String:
if validateUTF8 {
if pointer {
return sizeStringPtr, appendUTF8StringPtr
}
if slice {
return sizeStringSlice, appendUTF8StringSlice
}
if nozero {
return sizeStringValueNoZero, appendUTF8StringValueNoZero
}
return sizeStringValue, appendUTF8StringValue
}
if pointer {
return sizeStringPtr, appendStringPtr
}
@ -1984,9 +2016,6 @@ func appendBoolPackedSlice(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byt
}
func appendStringValue(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
v := *ptr.toString()
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
@ -1997,9 +2026,6 @@ func appendStringValueNoZero(b []byte, ptr pointer, wiretag uint64, _ bool) ([]b
if v == "" {
return b, nil
}
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
@ -2011,24 +2037,83 @@ func appendStringPtr(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, err
return b, nil
}
v := *p
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
return b, nil
}
func appendStringSlice(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
s := *ptr.toStringSlice()
for _, v := range s {
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
}
return b, nil
}
func appendUTF8StringValue(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
var invalidUTF8 bool
v := *ptr.toString()
if !utf8.ValidString(v) {
invalidUTF8 = true
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
if invalidUTF8 {
return b, errInvalidUTF8
}
return b, nil
}
func appendUTF8StringValueNoZero(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
var invalidUTF8 bool
v := *ptr.toString()
if v == "" {
return b, nil
}
if !utf8.ValidString(v) {
invalidUTF8 = true
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
if invalidUTF8 {
return b, errInvalidUTF8
}
return b, nil
}
func appendUTF8StringPtr(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
var invalidUTF8 bool
p := *ptr.toStringPtr()
if p == nil {
return b, nil
}
v := *p
if !utf8.ValidString(v) {
invalidUTF8 = true
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
if invalidUTF8 {
return b, errInvalidUTF8
}
return b, nil
}
func appendUTF8StringSlice(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
var invalidUTF8 bool
s := *ptr.toStringSlice()
for _, v := range s {
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
invalidUTF8 = true
}
b = appendVarint(b, wiretag)
b = appendVarint(b, uint64(len(v)))
b = append(b, v...)
}
if invalidUTF8 {
return b, errInvalidUTF8
}
return b, nil
}
func appendBytes(b []byte, ptr pointer, wiretag uint64, _ bool) ([]byte, error) {
@ -2107,7 +2192,8 @@ func makeGroupSliceMarshaler(u *marshalInfo) (sizer, marshaler) {
},
func(b []byte, ptr pointer, wiretag uint64, deterministic bool) ([]byte, error) {
s := ptr.getPointerSlice()
var err, errreq error
var err error
var nerr nonFatal
for _, v := range s {
if v.isNil() {
return b, errRepeatedHasNil
@ -2115,22 +2201,14 @@ func makeGroupSliceMarshaler(u *marshalInfo) (sizer, marshaler) {
b = appendVarint(b, wiretag) // start group
b, err = u.marshal(b, v, deterministic)
b = appendVarint(b, wiretag+(WireEndGroup-WireStartGroup)) // end group
if err != nil {
if _, ok := err.(*RequiredNotSetError); ok {
// Required field in submessage is not set.
// We record the error but keep going, to give a complete marshaling.
if errreq == nil {
errreq = err
}
continue
}
if !nerr.Merge(err) {
if err == ErrNil {
err = errRepeatedHasNil
}
return b, err
}
}
return b, errreq
return b, nerr.E
}
}
@ -2174,7 +2252,8 @@ func makeMessageSliceMarshaler(u *marshalInfo) (sizer, marshaler) {
},
func(b []byte, ptr pointer, wiretag uint64, deterministic bool) ([]byte, error) {
s := ptr.getPointerSlice()
var err, errreq error
var err error
var nerr nonFatal
for _, v := range s {
if v.isNil() {
return b, errRepeatedHasNil
@ -2184,22 +2263,14 @@ func makeMessageSliceMarshaler(u *marshalInfo) (sizer, marshaler) {
b = appendVarint(b, uint64(siz))
b, err = u.marshal(b, v, deterministic)
if err != nil {
if _, ok := err.(*RequiredNotSetError); ok {
// Required field in submessage is not set.
// We record the error but keep going, to give a complete marshaling.
if errreq == nil {
errreq = err
}
continue
}
if !nerr.Merge(err) {
if err == ErrNil {
err = errRepeatedHasNil
}
return b, err
}
}
return b, errreq
return b, nerr.E
}
}
@ -2223,14 +2294,33 @@ func makeMapMarshaler(f *reflect.StructField) (sizer, marshaler) {
// value.
// Key cannot be pointer-typed.
valIsPtr := valType.Kind() == reflect.Ptr
// If value is a message with nested maps, calling
// valSizer in marshal may be quadratic. We should use
// cached version in marshal (but not in size).
// If value is not message type, we don't have size cache,
// but it cannot be nested either. Just use valSizer.
valCachedSizer := valSizer
if valIsPtr && valType.Elem().Kind() == reflect.Struct {
u := getMarshalInfo(valType.Elem())
valCachedSizer = func(ptr pointer, tagsize int) int {
// Same as message sizer, but use cache.
p := ptr.getPointer()
if p.isNil() {
return 0
}
siz := u.cachedsize(p)
return siz + SizeVarint(uint64(siz)) + tagsize
}
}
return func(ptr pointer, tagsize int) int {
m := ptr.asPointerTo(t).Elem() // the map
n := 0
for _, k := range m.MapKeys() {
ki := k.Interface()
vi := m.MapIndex(k).Interface()
kaddr := toAddrPointer(&ki, false) // pointer to key
vaddr := toAddrPointer(&vi, valIsPtr) // pointer to value
kaddr := toAddrPointer(&ki, false, false) // pointer to key
vaddr := toAddrPointer(&vi, valIsPtr, false) // pointer to value
siz := keySizer(kaddr, 1) + valSizer(vaddr, 1) // tag of key = 1 (size=1), tag of val = 2 (size=1)
n += siz + SizeVarint(uint64(siz)) + tagsize
}
@ -2243,24 +2333,26 @@ func makeMapMarshaler(f *reflect.StructField) (sizer, marshaler) {
if len(keys) > 1 && deterministic {
sort.Sort(mapKeys(keys))
}
var nerr nonFatal
for _, k := range keys {
ki := k.Interface()
vi := m.MapIndex(k).Interface()
kaddr := toAddrPointer(&ki, false) // pointer to key
vaddr := toAddrPointer(&vi, valIsPtr) // pointer to value
kaddr := toAddrPointer(&ki, false, false) // pointer to key
vaddr := toAddrPointer(&vi, valIsPtr, false) // pointer to value
b = appendVarint(b, tag)
siz := keySizer(kaddr, 1) + valSizer(vaddr, 1) // tag of key = 1 (size=1), tag of val = 2 (size=1)
siz := keySizer(kaddr, 1) + valCachedSizer(vaddr, 1) // tag of key = 1 (size=1), tag of val = 2 (size=1)
b = appendVarint(b, uint64(siz))
b, err = keyMarshaler(b, kaddr, keyWireTag, deterministic)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
b, err = valMarshaler(b, vaddr, valWireTag, deterministic)
if err != nil && err != ErrNil { // allow nil value in map
if err != ErrNil && !nerr.Merge(err) { // allow nil value in map
return b, err
}
}
return b, nil
return b, nerr.E
}
}
@ -2316,7 +2408,7 @@ func (u *marshalInfo) sizeExtensions(ext *XXX_InternalExtensions) int {
// the last time this function was called.
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
n += ei.sizer(p, ei.tagsize)
}
mu.Unlock()
@ -2333,6 +2425,7 @@ func (u *marshalInfo) appendExtensions(b []byte, ext *XXX_InternalExtensions, de
defer mu.Unlock()
var err error
var nerr nonFatal
// Fast-path for common cases: zero or one extensions.
// Don't bother sorting the keys.
@ -2350,13 +2443,13 @@ func (u *marshalInfo) appendExtensions(b []byte, ext *XXX_InternalExtensions, de
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
b, err = ei.marshaler(b, p, ei.wiretag, deterministic)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
}
return b, nil
return b, nerr.E
}
// Sort the keys to provide a deterministic encoding.
@ -2381,13 +2474,13 @@ func (u *marshalInfo) appendExtensions(b []byte, ext *XXX_InternalExtensions, de
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
b, err = ei.marshaler(b, p, ei.wiretag, deterministic)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
}
return b, nil
return b, nerr.E
}
// message set format is:
@ -2426,7 +2519,7 @@ func (u *marshalInfo) sizeMessageSet(ext *XXX_InternalExtensions) int {
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
n += ei.sizer(p, 1) // message, tag = 3 (size=1)
}
mu.Unlock()
@ -2444,6 +2537,7 @@ func (u *marshalInfo) appendMessageSet(b []byte, ext *XXX_InternalExtensions, de
defer mu.Unlock()
var err error
var nerr nonFatal
// Fast-path for common cases: zero or one extensions.
// Don't bother sorting the keys.
@ -2468,14 +2562,14 @@ func (u *marshalInfo) appendMessageSet(b []byte, ext *XXX_InternalExtensions, de
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
b, err = ei.marshaler(b, p, 3<<3|WireBytes, deterministic)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
b = append(b, 1<<3|WireEndGroup)
}
return b, nil
return b, nerr.E
}
// Sort the keys to provide a deterministic encoding.
@ -2506,14 +2600,14 @@ func (u *marshalInfo) appendMessageSet(b []byte, ext *XXX_InternalExtensions, de
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
b, err = ei.marshaler(b, p, 3<<3|WireBytes, deterministic)
b = append(b, 1<<3|WireEndGroup)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
}
return b, nil
return b, nerr.E
}
// sizeV1Extensions computes the size of encoded data for a V1-API extension field.
@ -2536,7 +2630,7 @@ func (u *marshalInfo) sizeV1Extensions(m map[int32]Extension) int {
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
n += ei.sizer(p, ei.tagsize)
}
return n
@ -2556,6 +2650,7 @@ func (u *marshalInfo) appendV1Extensions(b []byte, m map[int32]Extension, determ
sort.Ints(keys)
var err error
var nerr nonFatal
for _, k := range keys {
e := m[int32(k)]
if e.value == nil || e.desc == nil {
@ -2570,13 +2665,13 @@ func (u *marshalInfo) appendV1Extensions(b []byte, m map[int32]Extension, determ
ei := u.getExtElemInfo(e.desc)
v := e.value
p := toAddrPointer(&v, ei.isptr)
p := toAddrPointer(&v, ei.isptr, ei.deref)
b, err = ei.marshaler(b, p, ei.wiretag, deterministic)
if err != nil {
if !nerr.Merge(err) {
return b, err
}
}
return b, nil
return b, nerr.E
}
// newMarshaler is the interface representing objects that can marshal themselves.

View file

@ -97,6 +97,8 @@ type unmarshalFieldInfo struct {
// if a required field, contains a single set bit at this field's index in the required field list.
reqMask uint64
name string // name of the field, for error reporting
}
var (
@ -134,10 +136,10 @@ func (u *unmarshalInfo) unmarshal(m pointer, b []byte) error {
u.computeUnmarshalInfo()
}
if u.isMessageSet {
return UnmarshalMessageSet(b, m.offset(u.extensions).toExtensions())
return unmarshalMessageSet(b, m.offset(u.extensions).toExtensions())
}
var reqMask uint64 // bitmask of required fields we've seen.
var rnse *RequiredNotSetError // an instance of a RequiredNotSetError returned by a submessage.
var reqMask uint64 // bitmask of required fields we've seen.
var errLater error
for len(b) > 0 {
// Read tag and wire type.
// Special case 1 and 2 byte varints.
@ -176,11 +178,20 @@ func (u *unmarshalInfo) unmarshal(m pointer, b []byte) error {
if r, ok := err.(*RequiredNotSetError); ok {
// Remember this error, but keep parsing. We need to produce
// a full parse even if a required field is missing.
rnse = r
if errLater == nil {
errLater = r
}
reqMask |= f.reqMask
continue
}
if err != errInternalBadWireType {
if err == errInvalidUTF8 {
if errLater == nil {
fullName := revProtoTypes[reflect.PtrTo(u.typ)] + "." + f.name
errLater = &invalidUTF8Error{fullName}
}
continue
}
return err
}
// Fragments with bad wire type are treated as unknown fields.
@ -239,20 +250,16 @@ func (u *unmarshalInfo) unmarshal(m pointer, b []byte) error {
emap[int32(tag)] = e
}
}
if rnse != nil {
// A required field of a submessage/group is missing. Return that error.
return rnse
}
if reqMask != u.reqMask {
if reqMask != u.reqMask && errLater == nil {
// A required field of this message is missing.
for _, n := range u.reqFields {
if reqMask&1 == 0 {
return &RequiredNotSetError{n}
errLater = &RequiredNotSetError{n}
}
reqMask >>= 1
}
}
return nil
return errLater
}
// computeUnmarshalInfo fills in u with information for use
@ -351,43 +358,52 @@ func (u *unmarshalInfo) computeUnmarshalInfo() {
}
// Store the info in the correct slot in the message.
u.setTag(tag, toField(&f), unmarshal, reqMask)
u.setTag(tag, toField(&f), unmarshal, reqMask, name)
}
// Find any types associated with oneof fields.
// TODO: XXX_OneofFuncs returns more info than we need. Get rid of some of it?
fn := reflect.Zero(reflect.PtrTo(t)).MethodByName("XXX_OneofFuncs")
if fn.IsValid() {
res := fn.Call(nil)[3] // last return value from XXX_OneofFuncs: []interface{}
for i := res.Len() - 1; i >= 0; i-- {
v := res.Index(i) // interface{}
tptr := reflect.ValueOf(v.Interface()).Type() // *Msg_X
typ := tptr.Elem() // Msg_X
var oneofImplementers []interface{}
switch m := reflect.Zero(reflect.PtrTo(t)).Interface().(type) {
case oneofFuncsIface:
_, _, _, oneofImplementers = m.XXX_OneofFuncs()
case oneofWrappersIface:
oneofImplementers = m.XXX_OneofWrappers()
}
for _, v := range oneofImplementers {
tptr := reflect.TypeOf(v) // *Msg_X
typ := tptr.Elem() // Msg_X
f := typ.Field(0) // oneof implementers have one field
baseUnmarshal := fieldUnmarshaler(&f)
tagstr := strings.Split(f.Tag.Get("protobuf"), ",")[1]
tag, err := strconv.Atoi(tagstr)
if err != nil {
panic("protobuf tag field not an integer: " + tagstr)
}
// Find the oneof field that this struct implements.
// Might take O(n^2) to process all of the oneofs, but who cares.
for _, of := range oneofFields {
if tptr.Implements(of.ityp) {
// We have found the corresponding interface for this struct.
// That lets us know where this struct should be stored
// when we encounter it during unmarshaling.
unmarshal := makeUnmarshalOneof(typ, of.ityp, baseUnmarshal)
u.setTag(tag, of.field, unmarshal, 0)
}
f := typ.Field(0) // oneof implementers have one field
baseUnmarshal := fieldUnmarshaler(&f)
tags := strings.Split(f.Tag.Get("protobuf"), ",")
fieldNum, err := strconv.Atoi(tags[1])
if err != nil {
panic("protobuf tag field not an integer: " + tags[1])
}
var name string
for _, tag := range tags {
if strings.HasPrefix(tag, "name=") {
name = strings.TrimPrefix(tag, "name=")
break
}
}
// Find the oneof field that this struct implements.
// Might take O(n^2) to process all of the oneofs, but who cares.
for _, of := range oneofFields {
if tptr.Implements(of.ityp) {
// We have found the corresponding interface for this struct.
// That lets us know where this struct should be stored
// when we encounter it during unmarshaling.
unmarshal := makeUnmarshalOneof(typ, of.ityp, baseUnmarshal)
u.setTag(fieldNum, of.field, unmarshal, 0, name)
}
}
}
// Get extension ranges, if any.
fn = reflect.Zero(reflect.PtrTo(t)).MethodByName("ExtensionRangeArray")
fn := reflect.Zero(reflect.PtrTo(t)).MethodByName("ExtensionRangeArray")
if fn.IsValid() {
if !u.extensions.IsValid() && !u.oldExtensions.IsValid() {
panic("a message with extensions, but no extensions field in " + t.Name())
@ -401,7 +417,7 @@ func (u *unmarshalInfo) computeUnmarshalInfo() {
// [0 0] is [tag=0/wiretype=varint varint-encoded-0].
u.setTag(0, zeroField, func(b []byte, f pointer, w int) ([]byte, error) {
return nil, fmt.Errorf("proto: %s: illegal tag 0 (wire type %d)", t, w)
}, 0)
}, 0, "")
// Set mask for required field check.
u.reqMask = uint64(1)<<uint(len(u.reqFields)) - 1
@ -413,8 +429,9 @@ func (u *unmarshalInfo) computeUnmarshalInfo() {
// tag = tag # for field
// field/unmarshal = unmarshal info for that field.
// reqMask = if required, bitmask for field position in required field list. 0 otherwise.
func (u *unmarshalInfo) setTag(tag int, field field, unmarshal unmarshaler, reqMask uint64) {
i := unmarshalFieldInfo{field: field, unmarshal: unmarshal, reqMask: reqMask}
// name = short name of the field.
func (u *unmarshalInfo) setTag(tag int, field field, unmarshal unmarshaler, reqMask uint64, name string) {
i := unmarshalFieldInfo{field: field, unmarshal: unmarshal, reqMask: reqMask, name: name}
n := u.typ.NumField()
if tag >= 0 && (tag < 16 || tag < 2*n) { // TODO: what are the right numbers here?
for len(u.dense) <= tag {
@ -442,11 +459,17 @@ func typeUnmarshaler(t reflect.Type, tags string) unmarshaler {
tagArray := strings.Split(tags, ",")
encoding := tagArray[0]
name := "unknown"
proto3 := false
validateUTF8 := true
for _, tag := range tagArray[3:] {
if strings.HasPrefix(tag, "name=") {
name = tag[5:]
}
if tag == "proto3" {
proto3 = true
}
}
validateUTF8 = validateUTF8 && proto3
// Figure out packaging (pointer, slice, or both)
slice := false
@ -594,6 +617,15 @@ func typeUnmarshaler(t reflect.Type, tags string) unmarshaler {
}
return unmarshalBytesValue
case reflect.String:
if validateUTF8 {
if pointer {
return unmarshalUTF8StringPtr
}
if slice {
return unmarshalUTF8StringSlice
}
return unmarshalUTF8StringValue
}
if pointer {
return unmarshalStringPtr
}
@ -1448,9 +1480,6 @@ func unmarshalStringValue(b []byte, f pointer, w int) ([]byte, error) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
*f.toString() = v
return b[x:], nil
}
@ -1468,9 +1497,6 @@ func unmarshalStringPtr(b []byte, f pointer, w int) ([]byte, error) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
*f.toStringPtr() = &v
return b[x:], nil
}
@ -1488,14 +1514,72 @@ func unmarshalStringSlice(b []byte, f pointer, w int) ([]byte, error) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
if !utf8.ValidString(v) {
return nil, errInvalidUTF8
}
s := f.toStringSlice()
*s = append(*s, v)
return b[x:], nil
}
func unmarshalUTF8StringValue(b []byte, f pointer, w int) ([]byte, error) {
if w != WireBytes {
return b, errInternalBadWireType
}
x, n := decodeVarint(b)
if n == 0 {
return nil, io.ErrUnexpectedEOF
}
b = b[n:]
if x > uint64(len(b)) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
*f.toString() = v
if !utf8.ValidString(v) {
return b[x:], errInvalidUTF8
}
return b[x:], nil
}
func unmarshalUTF8StringPtr(b []byte, f pointer, w int) ([]byte, error) {
if w != WireBytes {
return b, errInternalBadWireType
}
x, n := decodeVarint(b)
if n == 0 {
return nil, io.ErrUnexpectedEOF
}
b = b[n:]
if x > uint64(len(b)) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
*f.toStringPtr() = &v
if !utf8.ValidString(v) {
return b[x:], errInvalidUTF8
}
return b[x:], nil
}
func unmarshalUTF8StringSlice(b []byte, f pointer, w int) ([]byte, error) {
if w != WireBytes {
return b, errInternalBadWireType
}
x, n := decodeVarint(b)
if n == 0 {
return nil, io.ErrUnexpectedEOF
}
b = b[n:]
if x > uint64(len(b)) {
return nil, io.ErrUnexpectedEOF
}
v := string(b[:x])
s := f.toStringSlice()
*s = append(*s, v)
if !utf8.ValidString(v) {
return b[x:], errInvalidUTF8
}
return b[x:], nil
}
var emptyBuf [0]byte
func unmarshalBytesValue(b []byte, f pointer, w int) ([]byte, error) {
@ -1674,6 +1758,7 @@ func makeUnmarshalMap(f *reflect.StructField) unmarshaler {
// Maps will be somewhat slow. Oh well.
// Read key and value from data.
var nerr nonFatal
k := reflect.New(kt)
v := reflect.New(vt)
for len(b) > 0 {
@ -1694,7 +1779,7 @@ func makeUnmarshalMap(f *reflect.StructField) unmarshaler {
err = errInternalBadWireType // skip unknown tag
}
if err == nil {
if nerr.Merge(err) {
continue
}
if err != errInternalBadWireType {
@ -1717,7 +1802,7 @@ func makeUnmarshalMap(f *reflect.StructField) unmarshaler {
// Insert into map.
m.SetMapIndex(k.Elem(), v.Elem())
return r, nil
return r, nerr.E
}
}
@ -1743,15 +1828,16 @@ func makeUnmarshalOneof(typ, ityp reflect.Type, unmarshal unmarshaler) unmarshal
// Unmarshal data into holder.
// We unmarshal into the first field of the holder object.
var err error
var nerr nonFatal
b, err = unmarshal(b, valToPointer(v).offset(field0), w)
if err != nil {
if !nerr.Merge(err) {
return nil, err
}
// Write pointer to holder into target field.
f.asPointerTo(ityp).Elem().Set(v)
return b, nil
return b, nerr.E
}
}
@ -1864,7 +1950,7 @@ func encodeVarint(b []byte, x uint64) []byte {
// If there is an error, it returns 0,0.
func decodeVarint(b []byte) (uint64, int) {
var x, y uint64
if len(b) <= 0 {
if len(b) == 0 {
goto bad
}
x = uint64(b[0])

View file

@ -353,7 +353,7 @@ func (tm *TextMarshaler) writeStruct(w *textWriter, sv reflect.Value) error {
return err
}
}
if err := tm.writeAny(w, key, props.mkeyprop); err != nil {
if err := tm.writeAny(w, key, props.MapKeyProp); err != nil {
return err
}
if err := w.WriteByte('\n'); err != nil {
@ -370,7 +370,7 @@ func (tm *TextMarshaler) writeStruct(w *textWriter, sv reflect.Value) error {
return err
}
}
if err := tm.writeAny(w, val, props.mvalprop); err != nil {
if err := tm.writeAny(w, val, props.MapValProp); err != nil {
return err
}
if err := w.WriteByte('\n'); err != nil {

View file

@ -630,17 +630,17 @@ func (p *textParser) readStruct(sv reflect.Value, terminator string) error {
if err := p.consumeToken(":"); err != nil {
return err
}
if err := p.readAny(key, props.mkeyprop); err != nil {
if err := p.readAny(key, props.MapKeyProp); err != nil {
return err
}
if err := p.consumeOptionalSeparator(); err != nil {
return err
}
case "value":
if err := p.checkForColon(props.mvalprop, dst.Type().Elem()); err != nil {
if err := p.checkForColon(props.MapValProp, dst.Type().Elem()); err != nil {
return err
}
if err := p.readAny(val, props.mvalprop); err != nil {
if err := p.readAny(val, props.MapValProp); err != nil {
return err
}
if err := p.consumeOptionalSeparator(); err != nil {

View file

@ -130,10 +130,12 @@ func UnmarshalAny(any *any.Any, pb proto.Message) error {
// Is returns true if any value contains a given message type.
func Is(any *any.Any, pb proto.Message) bool {
aname, err := AnyMessageName(any)
if err != nil {
// The following is equivalent to AnyMessageName(any) == proto.MessageName(pb),
// but it avoids scanning TypeUrl for the slash.
if any == nil {
return false
}
return aname == proto.MessageName(pb)
name := proto.MessageName(pb)
prefix := len(any.TypeUrl) - len(name)
return prefix >= 1 && any.TypeUrl[prefix-1] == '/' && any.TypeUrl[prefix:] == name
}

View file

@ -1,11 +1,13 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: google/protobuf/any.proto
package any // import "github.com/golang/protobuf/ptypes/any"
package any
import proto "github.com/golang/protobuf/proto"
import fmt "fmt"
import math "math"
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
@ -16,7 +18,7 @@ var _ = math.Inf
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
// `Any` contains an arbitrary serialized protocol buffer message along with a
// URL that describes the type of the serialized message.
@ -99,17 +101,18 @@ const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
// }
//
type Any struct {
// A URL/resource name whose content describes the type of the
// serialized protocol buffer message.
// A URL/resource name that uniquely identifies the type of the serialized
// protocol buffer message. The last segment of the URL's path must represent
// the fully qualified name of the type (as in
// `path/google.protobuf.Duration`). The name should be in a canonical form
// (e.g., leading "." is not accepted).
//
// For URLs which use the scheme `http`, `https`, or no scheme, the
// following restrictions and interpretations apply:
// In practice, teams usually precompile into the binary all types that they
// expect it to use in the context of Any. However, for URLs which use the
// scheme `http`, `https`, or no scheme, one can optionally set up a type
// server that maps type URLs to message definitions as follows:
//
// * If no scheme is provided, `https` is assumed.
// * The last segment of the URL's path must represent the fully
// qualified name of the type (as in `path/google.protobuf.Duration`).
// The name should be in a canonical form (e.g., leading "." is
// not accepted).
// * An HTTP GET on the URL must yield a [google.protobuf.Type][]
// value in binary format, or produce an error.
// * Applications are allowed to cache lookup results based on the
@ -118,10 +121,14 @@ type Any struct {
// on changes to types. (Use versioned type names to manage
// breaking changes.)
//
// Note: this functionality is not currently available in the official
// protobuf release, and it is not used for type URLs beginning with
// type.googleapis.com.
//
// Schemes other than `http`, `https` (or the empty scheme) might be
// used with implementation specific semantics.
//
TypeUrl string `protobuf:"bytes,1,opt,name=type_url,json=typeUrl" json:"type_url,omitempty"`
TypeUrl string `protobuf:"bytes,1,opt,name=type_url,json=typeUrl,proto3" json:"type_url,omitempty"`
// Must be a valid serialized protocol buffer of the above specified type.
Value []byte `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
@ -133,17 +140,19 @@ func (m *Any) Reset() { *m = Any{} }
func (m *Any) String() string { return proto.CompactTextString(m) }
func (*Any) ProtoMessage() {}
func (*Any) Descriptor() ([]byte, []int) {
return fileDescriptor_any_744b9ca530f228db, []int{0}
return fileDescriptor_b53526c13ae22eb4, []int{0}
}
func (*Any) XXX_WellKnownType() string { return "Any" }
func (m *Any) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Any.Unmarshal(m, b)
}
func (m *Any) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Any.Marshal(b, m, deterministic)
}
func (dst *Any) XXX_Merge(src proto.Message) {
xxx_messageInfo_Any.Merge(dst, src)
func (m *Any) XXX_Merge(src proto.Message) {
xxx_messageInfo_Any.Merge(m, src)
}
func (m *Any) XXX_Size() int {
return xxx_messageInfo_Any.Size(m)
@ -172,9 +181,9 @@ func init() {
proto.RegisterType((*Any)(nil), "google.protobuf.Any")
}
func init() { proto.RegisterFile("google/protobuf/any.proto", fileDescriptor_any_744b9ca530f228db) }
func init() { proto.RegisterFile("google/protobuf/any.proto", fileDescriptor_b53526c13ae22eb4) }
var fileDescriptor_any_744b9ca530f228db = []byte{
var fileDescriptor_b53526c13ae22eb4 = []byte{
// 185 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4c, 0xcf, 0xcf, 0x4f,
0xcf, 0x49, 0xd5, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0x4f, 0x2a, 0x4d, 0xd3, 0x4f, 0xcc, 0xab, 0xd4,

View file

@ -82,7 +82,7 @@ func Duration(p *durpb.Duration) (time.Duration, error) {
return 0, fmt.Errorf("duration: %v is out of range for time.Duration", p)
}
if p.Nanos != 0 {
d += time.Duration(p.Nanos)
d += time.Duration(p.Nanos) * time.Nanosecond
if (d < 0) != (p.Nanos < 0) {
return 0, fmt.Errorf("duration: %v is out of range for time.Duration", p)
}

View file

@ -1,11 +1,13 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: google/protobuf/duration.proto
package duration // import "github.com/golang/protobuf/ptypes/duration"
package duration
import proto "github.com/golang/protobuf/proto"
import fmt "fmt"
import math "math"
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
@ -16,7 +18,7 @@ var _ = math.Inf
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
// A Duration represents a signed, fixed-length span of time represented
// as a count of seconds and fractions of seconds at nanosecond
@ -82,14 +84,14 @@ type Duration struct {
// Signed seconds of the span of time. Must be from -315,576,000,000
// to +315,576,000,000 inclusive. Note: these bounds are computed from:
// 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years
Seconds int64 `protobuf:"varint,1,opt,name=seconds" json:"seconds,omitempty"`
Seconds int64 `protobuf:"varint,1,opt,name=seconds,proto3" json:"seconds,omitempty"`
// Signed fractions of a second at nanosecond resolution of the span
// of time. Durations less than one second are represented with a 0
// `seconds` field and a positive or negative `nanos` field. For durations
// of one second or more, a non-zero value for the `nanos` field must be
// of the same sign as the `seconds` field. Must be from -999,999,999
// to +999,999,999 inclusive.
Nanos int32 `protobuf:"varint,2,opt,name=nanos" json:"nanos,omitempty"`
Nanos int32 `protobuf:"varint,2,opt,name=nanos,proto3" json:"nanos,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
@ -99,17 +101,19 @@ func (m *Duration) Reset() { *m = Duration{} }
func (m *Duration) String() string { return proto.CompactTextString(m) }
func (*Duration) ProtoMessage() {}
func (*Duration) Descriptor() ([]byte, []int) {
return fileDescriptor_duration_e7d612259e3f0613, []int{0}
return fileDescriptor_23597b2ebd7ac6c5, []int{0}
}
func (*Duration) XXX_WellKnownType() string { return "Duration" }
func (m *Duration) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Duration.Unmarshal(m, b)
}
func (m *Duration) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Duration.Marshal(b, m, deterministic)
}
func (dst *Duration) XXX_Merge(src proto.Message) {
xxx_messageInfo_Duration.Merge(dst, src)
func (m *Duration) XXX_Merge(src proto.Message) {
xxx_messageInfo_Duration.Merge(m, src)
}
func (m *Duration) XXX_Size() int {
return xxx_messageInfo_Duration.Size(m)
@ -138,11 +142,9 @@ func init() {
proto.RegisterType((*Duration)(nil), "google.protobuf.Duration")
}
func init() {
proto.RegisterFile("google/protobuf/duration.proto", fileDescriptor_duration_e7d612259e3f0613)
}
func init() { proto.RegisterFile("google/protobuf/duration.proto", fileDescriptor_23597b2ebd7ac6c5) }
var fileDescriptor_duration_e7d612259e3f0613 = []byte{
var fileDescriptor_23597b2ebd7ac6c5 = []byte{
// 190 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4b, 0xcf, 0xcf, 0x4f,
0xcf, 0x49, 0xd5, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0x4f, 0x2a, 0x4d, 0xd3, 0x4f, 0x29, 0x2d, 0x4a,

View file

@ -111,11 +111,9 @@ func TimestampNow() *tspb.Timestamp {
// TimestampProto converts the time.Time to a google.protobuf.Timestamp proto.
// It returns an error if the resulting Timestamp is invalid.
func TimestampProto(t time.Time) (*tspb.Timestamp, error) {
seconds := t.Unix()
nanos := int32(t.Sub(time.Unix(seconds, 0)))
ts := &tspb.Timestamp{
Seconds: seconds,
Nanos: nanos,
Seconds: t.Unix(),
Nanos: int32(t.Nanosecond()),
}
if err := validateTimestamp(ts); err != nil {
return nil, err

View file

@ -1,11 +1,13 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: google/protobuf/timestamp.proto
package timestamp // import "github.com/golang/protobuf/ptypes/timestamp"
package timestamp
import proto "github.com/golang/protobuf/proto"
import fmt "fmt"
import math "math"
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
@ -16,7 +18,7 @@ var _ = math.Inf
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package
// A Timestamp represents a point in time independent of any time zone
// or calendar, represented as seconds and fractions of seconds at
@ -81,7 +83,9 @@ const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
// {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional
// seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution),
// are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone
// is required, though only UTC (as indicated by "Z") is presently supported.
// is required. A proto3 JSON serializer should always use UTC (as indicated by
// "Z") when printing the Timestamp type and a proto3 JSON parser should be
// able to accept both UTC and other timezones (as indicated by an offset).
//
// For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past
// 01:30 UTC on January 15, 2017.
@ -92,20 +96,20 @@ const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
// to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime)
// with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one
// can use the Joda Time's [`ISODateTimeFormat.dateTime()`](
// http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--)
// to obtain a formatter capable of generating timestamps in this format.
// http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime--
// ) to obtain a formatter capable of generating timestamps in this format.
//
//
type Timestamp struct {
// Represents seconds of UTC time since Unix epoch
// 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
// 9999-12-31T23:59:59Z inclusive.
Seconds int64 `protobuf:"varint,1,opt,name=seconds" json:"seconds,omitempty"`
Seconds int64 `protobuf:"varint,1,opt,name=seconds,proto3" json:"seconds,omitempty"`
// Non-negative fractions of a second at nanosecond resolution. Negative
// second values with fractions must still have non-negative nanos values
// that count forward in time. Must be from 0 to 999,999,999
// inclusive.
Nanos int32 `protobuf:"varint,2,opt,name=nanos" json:"nanos,omitempty"`
Nanos int32 `protobuf:"varint,2,opt,name=nanos,proto3" json:"nanos,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
@ -115,17 +119,19 @@ func (m *Timestamp) Reset() { *m = Timestamp{} }
func (m *Timestamp) String() string { return proto.CompactTextString(m) }
func (*Timestamp) ProtoMessage() {}
func (*Timestamp) Descriptor() ([]byte, []int) {
return fileDescriptor_timestamp_b826e8e5fba671a8, []int{0}
return fileDescriptor_292007bbfe81227e, []int{0}
}
func (*Timestamp) XXX_WellKnownType() string { return "Timestamp" }
func (m *Timestamp) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Timestamp.Unmarshal(m, b)
}
func (m *Timestamp) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Timestamp.Marshal(b, m, deterministic)
}
func (dst *Timestamp) XXX_Merge(src proto.Message) {
xxx_messageInfo_Timestamp.Merge(dst, src)
func (m *Timestamp) XXX_Merge(src proto.Message) {
xxx_messageInfo_Timestamp.Merge(m, src)
}
func (m *Timestamp) XXX_Size() int {
return xxx_messageInfo_Timestamp.Size(m)
@ -154,11 +160,9 @@ func init() {
proto.RegisterType((*Timestamp)(nil), "google.protobuf.Timestamp")
}
func init() {
proto.RegisterFile("google/protobuf/timestamp.proto", fileDescriptor_timestamp_b826e8e5fba671a8)
}
func init() { proto.RegisterFile("google/protobuf/timestamp.proto", fileDescriptor_292007bbfe81227e) }
var fileDescriptor_timestamp_b826e8e5fba671a8 = []byte{
var fileDescriptor_292007bbfe81227e = []byte{
// 191 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4f, 0xcf, 0xcf, 0x4f,
0xcf, 0x49, 0xd5, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0x4f, 0x2a, 0x4d, 0xd3, 0x2f, 0xc9, 0xcc, 0x4d,