Class TokenBuffer

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, Versioned

    public class TokenBuffer
    extends JsonGenerator
    Utility class used for efficient storage of JsonToken sequences, needed for temporary buffering. Space efficient for different sequence lengths (especially so for smaller ones; but not significantly less efficient for larger), highly efficient for linear iteration and appending. Implemented as segmented/chunked linked list of tokens; only modifications are via appends.
    Since:
    1.5
    • Field Detail

      • DEFAULT_PARSER_FEATURES

        protected static final int DEFAULT_PARSER_FEATURES
      • _objectCodec

        protected ObjectCodec _objectCodec
        Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.
      • _generatorFeatures

        protected int _generatorFeatures
        Bit flag composed of bits that indicate which JsonGenerator.Features are enabled.

        NOTE: most features have no effect on this class

      • _closed

        protected boolean _closed
      • _last

        protected TokenBuffer.Segment _last
        Last segment of this buffer, one that is used for appending more tokens
      • _appendOffset

        protected int _appendOffset
        Offset within last segment,
    • Constructor Detail

      • TokenBuffer

        public TokenBuffer​(ObjectCodec codec)
        Parameters:
        codec - Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.
    • Method Detail

      • asParser

        public JsonParser asParser()
        Method used to create a JsonParser that can read contents stored in this buffer. Will use default _objectCodec for object conversions.

        Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

        Returns:
        Parser that can be used for reading contents stored in this buffer
      • asParser

        public JsonParser asParser​(ObjectCodec codec)
        Method used to create a JsonParser that can read contents stored in this buffer.

        Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

        Parameters:
        codec - Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.
        Returns:
        Parser that can be used for reading contents stored in this buffer
      • asParser

        public JsonParser asParser​(JsonParser src)
        Parameters:
        src - Parser to use for accessing source information like location, configured codec
      • serialize

        public void serialize​(JsonGenerator jgen)
                       throws java.io.IOException,
                              JsonGenerationException
        Helper method that will write all contents of this buffer using given JsonGenerator.

        Note: this method would be enough to implement JsonSerializer for TokenBuffer type; but we can not have upwards references (from core to mapper package); and as such we also can not take second argument.

        Throws:
        java.io.IOException
        JsonGenerationException
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • getOutputContext

        public final JsonWriteContext getOutputContext()
        Specified by:
        getOutputContext in class JsonGenerator
        Returns:
        Context object that can give information about logical position within generated json content.
      • flush

        public void flush()
                   throws java.io.IOException
        Description copied from class: JsonGenerator
        Method called to flush any buffered content to the underlying target (output stream, writer), and to flush the target itself as well.
        Specified by:
        flush in class JsonGenerator
        Throws:
        java.io.IOException
      • close

        public void close()
                   throws java.io.IOException
        Description copied from class: JsonGenerator
        Method called to close this generator, so that no more content can be written.

        Whether the underlying target (stream, writer) gets closed depends on whether this generator either manages the target (i.e. is the only one with access to the target -- case if caller passes a reference to the resource such as File, but not stream); or has feature JsonGenerator.Feature.AUTO_CLOSE_TARGET enabled. If either of above is true, the target is also closed. Otherwise (not managing, feature not enabled), target is not closed.

        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in class JsonGenerator
        Throws:
        java.io.IOException
      • isClosed

        public boolean isClosed()
        Description copied from class: JsonGenerator
        Method that can be called to determine whether this generator is closed or not. If it is closed, no more output can be done.
        Specified by:
        isClosed in class JsonGenerator
      • writeStartArray

        public final void writeStartArray()
                                   throws java.io.IOException,
                                          JsonGenerationException
        Description copied from class: JsonGenerator
        Method for writing starting marker of a JSON Array value (character '['; plus possible white space decoration if pretty-printing is enabled).

        Array values can be written in any context where values are allowed: meaning everywhere except for when a field name is expected.

        Specified by:
        writeStartArray in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeEndArray

        public final void writeEndArray()
                                 throws java.io.IOException,
                                        JsonGenerationException
        Description copied from class: JsonGenerator
        Method for writing closing marker of a JSON Array value (character ']'; plus possible white space decoration if pretty-printing is enabled).

        Marker can be written if the innermost structured type is Array.

        Specified by:
        writeEndArray in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeStartObject

        public final void writeStartObject()
                                    throws java.io.IOException,
                                           JsonGenerationException
        Description copied from class: JsonGenerator
        Method for writing starting marker of a JSON Object value (character '{'; plus possible white space decoration if pretty-printing is enabled).

        Object values can be written in any context where values are allowed: meaning everywhere except for when a field name is expected.

        Specified by:
        writeStartObject in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeEndObject

        public final void writeEndObject()
                                  throws java.io.IOException,
                                         JsonGenerationException
        Description copied from class: JsonGenerator
        Method for writing closing marker of a JSON Object value (character '}'; plus possible white space decoration if pretty-printing is enabled).

        Marker can be written if the innermost structured type is Object, and the last written event was either a complete value, or START-OBJECT marker (see JSON specification for more details).

        Specified by:
        writeEndObject in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeFieldName

        public final void writeFieldName​(java.lang.String name)
                                  throws java.io.IOException,
                                         JsonGenerationException
        Description copied from class: JsonGenerator
        Method for writing a field name (JSON String surrounded by double quotes: syntactically identical to a JSON String value), possibly decorated by white space if pretty-printing is enabled.

        Field names can only be written in Object context (check out JSON specification for details), when field name is expected (field names alternate with values).

        Specified by:
        writeFieldName in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeFieldName

        public void writeFieldName​(SerializableString name)
                            throws java.io.IOException,
                                   JsonGenerationException
        Description copied from class: JsonGenerator
        Method similar to JsonGenerator.writeFieldName(String), main difference being that it may perform better as some of processing (such as quoting of certain characters, or encoding into external encoding if supported by generator) can be done just once and reused for later calls.

        Default implementation simple uses unprocessed name container in serialized String; implementations are strongly encouraged to make use of more efficient methods argument object has.

        Overrides:
        writeFieldName in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeFieldName

        public void writeFieldName​(SerializedString name)
                            throws java.io.IOException,
                                   JsonGenerationException
        Description copied from class: JsonGenerator
        Method similar to JsonGenerator.writeFieldName(String), main difference being that it may perform better as some of processing (such as quoting of certain characters, or encoding into external encoding if supported by generator) can be done just once and reused for later calls.

        Default implementation simple uses unprocessed name container in serialized String; implementations are strongly encouraged to make use of more efficient methods argument object has.

        Overrides:
        writeFieldName in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeString

        public void writeString​(java.lang.String text)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting a String value. Depending on context this means either array element, (object) field value or a stand alone String; but in all cases, String will be surrounded in double quotes, and contents will be properly escaped as required by JSON specification.
        Specified by:
        writeString in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeString

        public void writeString​(char[] text,
                                int offset,
                                int len)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting a String value. Depending on context this means either array element, (object) field value or a stand alone String; but in all cases, String will be surrounded in double quotes, and contents will be properly escaped as required by JSON specification.
        Specified by:
        writeString in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRawUTF8String

        public void writeRawUTF8String​(byte[] text,
                                       int offset,
                                       int length)
                                throws java.io.IOException,
                                       JsonGenerationException
        Description copied from class: JsonGenerator
        Method similar to JsonGenerator.writeString(String) but that takes as its input a UTF-8 encoded String that is to be output as-is, without additional escaping (type of which depends on data format; backslashes for JSON). However, quoting that data format requires (like double-quotes for JSON) will be added around the value if and as necessary.

        Note that some backends may choose not to support this method: for example, if underlying destination is a Writer using this method would require UTF-8 decoding. If so, implementation may instead choose to throw a UnsupportedOperationException due to ineffectiveness of having to decode input.

        Specified by:
        writeRawUTF8String in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeUTF8String

        public void writeUTF8String​(byte[] text,
                                    int offset,
                                    int length)
                             throws java.io.IOException,
                                    JsonGenerationException
        Description copied from class: JsonGenerator
        Method similar to JsonGenerator.writeString(String) but that takes as its input a UTF-8 encoded String which has not been escaped using whatever escaping scheme data format requires (for JSON that is backslash-escaping for control characters and double-quotes; for other formats something else). This means that textual JSON backends need to check if value needs JSON escaping, but otherwise can just be copied as is to output. Also, quoting that data format requires (like double-quotes for JSON) will be added around the value if and as necessary.

        Note that some backends may choose not to support this method: for example, if underlying destination is a Writer using this method would require UTF-8 decoding. In this case generator implementation may instead choose to throw a UnsupportedOperationException due to ineffectiveness of having to decode input.

        Specified by:
        writeUTF8String in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRaw

        public void writeRaw​(java.lang.String text)
                      throws java.io.IOException,
                             JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will force generator to copy input text verbatim with no modifications (including that no escaping is done and no separators are added even if context [array, object] would otherwise require such). If such separators are desired, use JsonGenerator.writeRawValue(String) instead.

        Note that not all generator implementations necessarily support such by-pass methods: those that do not will throw UnsupportedOperationException.

        Specified by:
        writeRaw in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRaw

        public void writeRaw​(java.lang.String text,
                             int offset,
                             int len)
                      throws java.io.IOException,
                             JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will force generator to copy input text verbatim with no modifications (including that no escaping is done and no separators are added even if context [array, object] would otherwise require such). If such separators are desired, use JsonGenerator.writeRawValue(String) instead.

        Note that not all generator implementations necessarily support such by-pass methods: those that do not will throw UnsupportedOperationException.

        Specified by:
        writeRaw in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRaw

        public void writeRaw​(char[] text,
                             int offset,
                             int len)
                      throws java.io.IOException,
                             JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will force generator to copy input text verbatim with no modifications (including that no escaping is done and no separators are added even if context [array, object] would otherwise require such). If such separators are desired, use JsonGenerator.writeRawValue(String) instead.

        Note that not all generator implementations necessarily support such by-pass methods: those that do not will throw UnsupportedOperationException.

        Specified by:
        writeRaw in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRaw

        public void writeRaw​(char c)
                      throws java.io.IOException,
                             JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will force generator to copy input text verbatim with no modifications (including that no escaping is done and no separators are added even if context [array, object] would otherwise require such). If such separators are desired, use JsonGenerator.writeRawValue(String) instead.

        Note that not all generator implementations necessarily support such by-pass methods: those that do not will throw UnsupportedOperationException.

        Specified by:
        writeRaw in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeRawValue

        public void writeRawValue​(java.lang.String text)
                           throws java.io.IOException,
                                  JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will force generator to copy input text verbatim without any modifications, but assuming it must constitute a single legal JSON value (number, string, boolean, null, Array or List). Assuming this, proper separators are added if and as needed (comma or colon), and generator state updated to reflect this.
        Specified by:
        writeRawValue in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(int i)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting given value as Json number. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(long l)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting given value as Json number. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(double d)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting indicate Json numeric value. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(float f)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting indicate Json numeric value. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(java.math.BigDecimal dec)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting indicate Json numeric value. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(java.math.BigInteger v)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting given value as Json number. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNumber

        public void writeNumber​(java.lang.String encodedValue)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Write method that can be used for custom numeric types that can not be (easily?) converted to "standard" Java number types. Because numbers are not surrounded by double quotes, regular JsonGenerator.writeString(java.lang.String) method can not be used; nor JsonGenerator.writeRaw(java.lang.String) because that does not properly handle value separators needed in Array or Object contexts.

        Note: because of lack of type safety, some generator implementations may not be able to implement this method. For example, if a binary json format is used, it may require type information for encoding; similarly for generator-wrappers around Java objects or Json nodes. If implementation does not implement this method, it needs to throw UnsupportedOperationException.

        Specified by:
        writeNumber in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeBoolean

        public void writeBoolean​(boolean state)
                          throws java.io.IOException,
                                 JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting literal Json boolean value (one of Strings 'true' and 'false'). Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeBoolean in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeNull

        public void writeNull()
                       throws java.io.IOException,
                              JsonGenerationException
        Description copied from class: JsonGenerator
        Method for outputting literal Json null value. Can be called in any context where a value is expected (Array value, Object field value, root-level value). Additional white space may be added around the value if pretty-printing is enabled.
        Specified by:
        writeNull in class JsonGenerator
        Throws:
        java.io.IOException
        JsonGenerationException
      • writeObject

        public void writeObject​(java.lang.Object value)
                         throws java.io.IOException,
                                JsonProcessingException
        Description copied from class: JsonGenerator
        Method for writing given Java object (POJO) as Json. Exactly how the object gets written depends on object in question (ad on codec, its configuration); for most beans it will result in Json object, but for others Json array, or String or numeric value (and for nulls, Json null literal. NOTE: generator must have its object codec set to non-null value; for generators created by a mapping factory this is the case, for others not.
        Specified by:
        writeObject in class JsonGenerator
        Throws:
        java.io.IOException
        JsonProcessingException
      • writeBinary

        public void writeBinary​(Base64Variant b64variant,
                                byte[] data,
                                int offset,
                                int len)
                         throws java.io.IOException,
                                JsonGenerationException
        Description copied from class: JsonGenerator
        Method that will output given chunk of binary data as base64 encoded, as a complete String value (surrounded by double quotes). This method defaults

        Note: because Json Strings can not contain unescaped linefeeds, if linefeeds are included (as per last argument), they must be escaped. This adds overhead for decoding without improving readability. Alternatively if linefeeds are not included, resulting String value may violate the requirement of base64 RFC which mandates line-length of 76 characters and use of linefeeds. However, all JsonParser implementations are required to accept such "long line base64"; as do typical production-level base64 decoders.

        Specified by:
        writeBinary in class JsonGenerator
        Parameters:
        b64variant - Base64 variant to use: defines details such as whether padding is used (and if so, using which character); what is the maximum line length before adding linefeed, and also the underlying alphabet to use.
        Throws:
        java.io.IOException
        JsonGenerationException
      • copyCurrentEvent

        public void copyCurrentEvent​(JsonParser jp)
                              throws java.io.IOException,
                                     JsonProcessingException
        Description copied from class: JsonGenerator
        Method for copying contents of the current event that the given parser instance points to. Note that the method will not copy any other events, such as events contained within Json Array or Object structures.

        Calling this method will not advance the given parser, although it may cause parser to internally process more data (if it lazy loads contents of value events, for example)

        Specified by:
        copyCurrentEvent in class JsonGenerator
        Throws:
        java.io.IOException
        JsonProcessingException
      • copyCurrentStructure

        public void copyCurrentStructure​(JsonParser jp)
                                  throws java.io.IOException,
                                         JsonProcessingException
        Description copied from class: JsonGenerator
        Method for copying contents of the current event and following events that it encloses the given parser instance points to.

        So what constitutes enclosing? Here is the list of events that have associated enclosed events that will get copied:

        After calling this method, parser will point to the last event that was copied. This will either be the event parser already pointed to (if there were no enclosed events), or the last enclosed event copied.

        Specified by:
        copyCurrentStructure in class JsonGenerator
        Throws:
        java.io.IOException
        JsonProcessingException
      • _append

        protected final void _append​(JsonToken type)
      • _append

        protected final void _append​(JsonToken type,
                                     java.lang.Object value)
      • _reportUnsupportedOperation

        protected void _reportUnsupportedOperation()