a1ffe78fc7
feat(api): Add image multimodal support for LLMNode ( #17372 )
...
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.
This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.
To integrate file outputs into workflows, adjustments to the frontend code
are necessary.
For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.
Close #15814 .
Signed-off-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: -LAN- <laipz8200@outlook.com >
2025-04-30 17:28:02 +08:00
af7a907960
immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer ( #17957 )
2025-04-30 16:07:20 +08:00
487b107e6f
Enhance Code Consistency Across Repository with .editorconfig ( #19023 )
2025-04-29 18:04:33 +08:00
63802d8104
refactor: rename plugin manager to plugin client and rename path from manager to impl ( #18876 )
2025-04-27 14:22:25 +08:00
84b076bf93
fix: LLMResultChunk cause concatenate str and list exception ( #18852 )
2025-04-27 11:32:14 +08:00
cabcaf7439
fix : #18744 The model order defined in position.yaml in the Model Plugin is not taking effect. ( #18756 )
2025-04-25 16:45:48 +08:00
8ebdd4649c
fix: Update prompt message content types to use Literal and add union type for content ( #17136 )
...
Co-authored-by: 朱庆超 <zhuqingchao@xiaomi.com >
Co-authored-by: crazywoola <427733928@qq.com >
2025-04-22 16:17:55 +08:00
e31c98eea3
fix(promptMessage): correct field_serializer implementation for content serialization ( #18458 )
2025-04-21 15:09:49 +08:00
2eb79ef233
feat: structured output ( #17877 )
2025-04-18 16:33:53 +08:00
491a1beed3
fix: invalid new tool call creation logic during response handling in OAI-Compat model ( #17781 )
2025-04-17 16:52:49 +08:00
93eef1625f
feat: re-add prompt messages to result and chunks in llm ( #17883 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-04-11 17:04:49 +08:00
27edcb6e7c
fix: autocorrect everything in api ( #17859 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2025-04-11 15:24:39 +08:00
bf11af4e04
feat(large_language_model): Adds plugin-based token counting configuration option ( #17706 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: Yeuoly <admin@srmxy.cn >
2025-04-09 20:52:58 +08:00
44f911a0a8
chore: docstring not match the function parameter ( #17162 )
2025-03-31 13:19:15 +08:00
8047d08b3b
chore: remove third party models ( #13885 )
2025-03-27 10:22:32 +08:00
a30945312a
fix: typos ( #16385 )
2025-03-21 11:14:40 +08:00
963b6f628a
Chore: PromptMessage is not an abstract base class ( #15965 )
2025-03-18 10:57:52 +08:00
037f200527
fix: invoke_error is not callable ( #15555 )
...
Co-authored-by: 刘江波 <jiangbo721@163.com >
2025-03-12 10:58:44 +08:00
4668c4996a
feat: Add caching mechanism for plugin model schemas ( #14898 )
2025-03-04 18:02:06 +08:00
7259c0d69f
fix: fix a typo of get_customizable_model_schema method name ( #14449 )
2025-03-01 22:56:13 +08:00
490b6d092e
Fix/plugin race condition ( #14253 )
2025-02-25 12:20:47 +08:00
f535a2aa71
chore: prompt_message is actually assistant_message which is a bit am… ( #13839 )
...
Co-authored-by: 刘江波 <jiangbo721@163.com >
2025-02-19 09:14:10 +08:00
403e2d58b9
Introduce Plugins ( #13836 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
Signed-off-by: -LAN- <laipz8200@outlook.com >
Signed-off-by: xhe <xw897002528@gmail.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: takatost <takatost@gmail.com >
Co-authored-by: kurokobo <kuro664@gmail.com >
Co-authored-by: Novice Lee <novicelee@NoviPro.local >
Co-authored-by: zxhlyh <jasonapring2015@outlook.com >
Co-authored-by: AkaraChen <akarachen@outlook.com >
Co-authored-by: Yi <yxiaoisme@gmail.com >
Co-authored-by: Joel <iamjoel007@gmail.com >
Co-authored-by: JzoNg <jzongcode@gmail.com >
Co-authored-by: twwu <twwu@dify.ai >
Co-authored-by: Hiroshi Fujita <fujita-h@users.noreply.github.com >
Co-authored-by: AkaraChen <85140972+AkaraChen@users.noreply.github.com >
Co-authored-by: NFish <douxc512@gmail.com >
Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com >
Co-authored-by: 非法操作 <hjlarry@163.com >
Co-authored-by: Novice <857526207@qq.com >
Co-authored-by: Hiroki Nagai <82458324+nagaihiroki-git@users.noreply.github.com >
Co-authored-by: Gen Sato <52241300+halogen22@users.noreply.github.com >
Co-authored-by: eux <euxuuu@gmail.com >
Co-authored-by: huangzhuo1949 <167434202+huangzhuo1949@users.noreply.github.com >
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com >
Co-authored-by: lotsik <lotsik@mail.ru >
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
Co-authored-by: nite-knite <nkCoding@gmail.com >
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com >
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gakkiyomi <gakkiyomi@aliyun.com >
Co-authored-by: CN-P5 <heibai2006@gmail.com >
Co-authored-by: CN-P5 <heibai2006@qq.com >
Co-authored-by: Chuehnone <1897025+chuehnone@users.noreply.github.com >
Co-authored-by: yihong <zouzou0208@gmail.com >
Co-authored-by: Kevin9703 <51311316+Kevin9703@users.noreply.github.com >
Co-authored-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: Boris Feld <lothiraldan@gmail.com >
Co-authored-by: mbo <himabo@gmail.com >
Co-authored-by: mabo <mabo@aeyes.ai >
Co-authored-by: Warren Chen <warren.chen830@gmail.com >
Co-authored-by: JzoNgKVO <27049666+JzoNgKVO@users.noreply.github.com >
Co-authored-by: jiandanfeng <chenjh3@wangsu.com >
Co-authored-by: zhu-an <70234959+xhdd123321@users.noreply.github.com >
Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com >
Co-authored-by: 海狸大師 <86974027+yenslife@users.noreply.github.com >
Co-authored-by: Xu Song <xusong.vip@gmail.com >
Co-authored-by: rayshaw001 <396301947@163.com >
Co-authored-by: Ding Jiatong <dingjiatong@gmail.com >
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn >
Co-authored-by: JasonVV <jasonwangiii@outlook.com >
Co-authored-by: le0zh <newlight@qq.com >
Co-authored-by: zhuxinliang <zhuxinliang@didiglobal.com >
Co-authored-by: k-zaku <zaku99@outlook.jp >
Co-authored-by: luckylhb90 <luckylhb90@gmail.com >
Co-authored-by: hobo.l <hobo.l@binance.com >
Co-authored-by: jiangbo721 <365065261@qq.com >
Co-authored-by: 刘江波 <jiangbo721@163.com >
Co-authored-by: Shun Miyazawa <34241526+miya@users.noreply.github.com >
Co-authored-by: EricPan <30651140+Egfly@users.noreply.github.com >
Co-authored-by: crazywoola <427733928@qq.com >
Co-authored-by: sino <sino2322@gmail.com >
Co-authored-by: Jhvcc <37662342+Jhvcc@users.noreply.github.com >
Co-authored-by: lowell <lowell.hu@zkteco.in >
Co-authored-by: Boris Polonsky <BorisPolonsky@users.noreply.github.com >
Co-authored-by: Ademílson Tonato <ademilsonft@outlook.com >
Co-authored-by: Ademílson Tonato <ademilson.tonato@refurbed.com >
Co-authored-by: IWAI, Masaharu <iwaim.sub@gmail.com >
Co-authored-by: Yueh-Po Peng (Yabi) <94939112+y10ab1@users.noreply.github.com >
Co-authored-by: Jason <ggbbddjm@gmail.com >
Co-authored-by: Xin Zhang <sjhpzx@gmail.com >
Co-authored-by: yjc980121 <3898524+yjc980121@users.noreply.github.com >
Co-authored-by: heyszt <36215648+hieheihei@users.noreply.github.com >
Co-authored-by: Abdullah AlOsaimi <osaimiacc@gmail.com >
Co-authored-by: Abdullah AlOsaimi <189027247+osaimi@users.noreply.github.com >
Co-authored-by: Yingchun Lai <laiyingchun@apache.org >
Co-authored-by: Hash Brown <hi@xzd.me >
Co-authored-by: zuodongxu <192560071+zuodongxu@users.noreply.github.com >
Co-authored-by: Masashi Tomooka <tmokmss@users.noreply.github.com >
Co-authored-by: aplio <ryo.091219@gmail.com >
Co-authored-by: Obada Khalili <54270856+obadakhalili@users.noreply.github.com >
Co-authored-by: Nam Vu <zuzoovn@gmail.com >
Co-authored-by: Kei YAMAZAKI <1715090+kei-yamazaki@users.noreply.github.com >
Co-authored-by: TechnoHouse <13776377+deephbz@users.noreply.github.com >
Co-authored-by: Riddhimaan-Senapati <114703025+Riddhimaan-Senapati@users.noreply.github.com >
Co-authored-by: MaFee921 <31881301+2284730142@users.noreply.github.com >
Co-authored-by: te-chan <t-nakanome@sakura-is.co.jp >
Co-authored-by: HQidea <HQidea@users.noreply.github.com >
Co-authored-by: Joshbly <36315710+Joshbly@users.noreply.github.com >
Co-authored-by: xhe <xw897002528@gmail.com >
Co-authored-by: weiwenyan-dev <154779315+weiwenyan-dev@users.noreply.github.com >
Co-authored-by: ex_wenyan.wei <ex_wenyan.wei@tcl.com >
Co-authored-by: engchina <12236799+engchina@users.noreply.github.com >
Co-authored-by: engchina <atjapan2015@gmail.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 呆萌闷油瓶 <253605712@qq.com >
Co-authored-by: Kemal <kemalmeler@outlook.com >
Co-authored-by: Lazy_Frog <4590648+lazyFrogLOL@users.noreply.github.com >
Co-authored-by: Yi Xiao <54782454+YIXIAO0@users.noreply.github.com >
Co-authored-by: Steven sun <98230804+Tuyohai@users.noreply.github.com >
Co-authored-by: steven <sunzwj@digitalchina.com >
Co-authored-by: Kalo Chin <91766386+fdb02983rhy@users.noreply.github.com >
Co-authored-by: Katy Tao <34019945+KatyTao@users.noreply.github.com >
Co-authored-by: depy <42985524+h4ckdepy@users.noreply.github.com >
Co-authored-by: 胡春东 <gycm520@gmail.com >
Co-authored-by: Junjie.M <118170653@qq.com >
Co-authored-by: MuYu <mr.muzea@gmail.com >
Co-authored-by: Naoki Takashima <39912547+takatea@users.noreply.github.com >
Co-authored-by: Summer-Gu <37869445+gubinjie@users.noreply.github.com >
Co-authored-by: Fei He <droxer.he@gmail.com >
Co-authored-by: ybalbert001 <120714773+ybalbert001@users.noreply.github.com >
Co-authored-by: Yuanbo Li <ybalbert@amazon.com >
Co-authored-by: douxc <7553076+douxc@users.noreply.github.com >
Co-authored-by: liuzhenghua <1090179900@qq.com >
Co-authored-by: Wu Jiayang <62842862+Wu-Jiayang@users.noreply.github.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: kimjion <45935338+kimjion@users.noreply.github.com >
Co-authored-by: AugNSo <song.tiankai@icloud.com >
Co-authored-by: llinvokerl <38915183+llinvokerl@users.noreply.github.com >
Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com >
Co-authored-by: Vasu Negi <vasu-negi@users.noreply.github.com >
Co-authored-by: Hundredwz <1808096180@qq.com >
Co-authored-by: Xiyuan Chen <52963600+GareArc@users.noreply.github.com >
2025-02-17 17:05:13 +08:00
a3d3e30e3a
fix: fix tongyi models blocking mode with incremental_output=stream ( #13620 )
2025-02-13 10:24:05 +08:00
1f38d4846b
fix: issue #13483 and #13434 ( #13518 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2025-02-11 12:45:49 +08:00
c8357da13b
[Fix] Sagemaker LLM Provider can't adjust context size, it'a always 2… ( #13462 )
...
Co-authored-by: Yuanbo Li <ybalbert@amazon.com >
2025-02-10 12:25:04 +08:00
7796984444
Fix: Removed model params except max_token for deepseek r1 in volcengine ( #13446 )
2025-02-10 10:26:26 +08:00
75113c26c6
Feat : add deepseek support for tongyi ( #13445 )
2025-02-10 10:26:03 +08:00
939a9ecd21
chore: use the wrap thinking api for volcengine ( #13432 )
...
Signed-off-by: xhe <xw897002528@gmail.com >
2025-02-10 10:25:07 +08:00
811d72a727
feat: added a _position.yaml for vertex ai provider ( #13367 )
2025-02-09 10:29:07 +08:00
37117c22d4
feat(model): support Gemini 2.0 Flash Lite Preview model (02-05) in Google's model provider ( #13399 )
2025-02-09 10:22:33 +08:00
286cdc41ab
reasoning model unified think tag is <think></think> ( #13392 )
...
Co-authored-by: crazywoola <427733928@qq.com >
2025-02-08 16:19:41 +08:00
c8dcde6cd0
fix: Gemini 2.0 Flash 001 model yaml file naming ( #13372 )
2025-02-08 09:12:42 +08:00
8f9db61688
feat: added new silicon flow models ( #13369 )
2025-02-08 09:12:22 +08:00
38c31e64db
add enable_search parameter to qwen_max, plus, turbo ( #13335 )
...
Co-authored-by: steven <sunzwj@digitalchina.com >
2025-02-07 22:16:26 +08:00
413dfd5628
feat: add completion mode and context size options for LLM configuration ( #13325 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-07 15:08:53 +08:00
f9515901cc
fix: Azure AI Foundry model cannot be used in the workflow ( #13323 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-07 14:52:57 +08:00
3f42fabff8
chore:improve thinking display for llm from xinference and ollama pro… ( #13318 )
2025-02-07 14:29:29 +08:00
1caa578771
chore(*): Update style of thinking ( #13319 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-07 14:06:35 +08:00
3eb3db0663
chore: refactor the OpenAICompatible and improve thinking display ( #13299 )
2025-02-07 13:28:46 +08:00
6e5c915f96
feat(model): add deepseek-r1 for openrouter ( #13312 )
2025-02-07 12:39:13 +08:00
2348abe4bf
feat: added a couple of models not defined in vertex ai, that were already … ( #13296 )
2025-02-07 09:11:25 +08:00
f7e7a399d9
feat:add think tag display for xinference deepseek r1 ( #13291 )
2025-02-06 22:04:58 +08:00
16865d43a8
feat: add deepseek models for volcengine provider ( #13283 )
...
Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com >
2025-02-06 18:20:03 +08:00
0d13aee15c
feat:add deepseek r1 think display for ollama provider ( #13272 )
2025-02-06 15:32:10 +08:00
40dd63ecef
Upgrade oracle models ( #13174 )
...
Co-authored-by: engchina <atjapan2015@gmail.com >
2025-02-06 13:24:27 +08:00
6d66d6da15
feat(model_providers): Support deepseek-r1 for Nvidia Catalog ( #13269 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-06 13:03:19 +08:00
87763fc234
feat(model_providers): Support deepseek for Azure AI Foundry ( #13267 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-06 12:45:48 +08:00
f6c44cae2e
feat(model): add gemini-2.0 model ( #13266 )
2025-02-06 12:28:59 +08:00
da2ee04fce
fix: correct linewrap think display in generic openai api ( #13260 )
...
Signed-off-by: xhe <xw897002528@gmail.com >
2025-02-06 10:53:08 +08:00
QuantumGhost