To be used in langchain
How does this model or its smaller variants fair when used in langchain?
How important is it (if not used in langchain) to use the format below (also mentioned in model card)?
BEGININPUT - denotes a new input block
BEGINCONTEXT - denotes the block of context (metadata key/value pairs) to associate with the current input block
ENDCONTEXT - denotes the end of the metadata block for the current input
[text] - Insert whatever text you want for the input block, as many paragraphs as can fit in the context.
ENDINPUT - denotes the end of the current input block
[repeat as many input blocks in this format as you want]
BEGININSTRUCTION - denotes the start of the list (or one) instruction(s) to respond to for all of the input blocks above.
[instruction(s)]
ENDINSTRUCTION - denotes the end of instruction set
The contextual question answering format was added as a way to enhance performance of retrieval augmented generation - the idea being you fetch one or more context blocks via similarity search (or whatever retrieval mechanism), with some metadata (e.g. url/date/author as a source of truth mechanism to reference) and the answer should better align with the context.
The training data includes examples with up to 5 input blocks with one or more questions/instructions that make use of 1 or more of the input blocks. It may not be required for question answering, but it does seem to help. I have not used the model with langchain so I can't say for sure.