[Bugfix] catch AssertionError in MistralTokenizer as ValueError (#16344)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
This commit is contained in:
parent
a25866ac8d
commit
c3b5189137
@ -1193,8 +1193,15 @@ def apply_mistral_chat_template(
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
try:
|
||||
return tokenizer.apply_chat_template(
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
**kwargs,
|
||||
)
|
||||
# mistral-common uses assert statements to stop processing of input
|
||||
# if input does not comply with the expected format.
|
||||
# We convert those assertion errors to ValueErrors so they can be
|
||||
# are properly caught in the preprocessing_input step
|
||||
except AssertionError as e:
|
||||
raise ValueError from e
|
||||
|
Loading…
x
Reference in New Issue
Block a user