All issues

Search results

decoder-only sink mask update policy

Status: Open.

#48 In tomaarsen/attention_sinks;

· ZJLi2013 opened on Jul 1, 2025

TypeError: bad operand type for unary -: 'NoneType'

Status: Open.

#47 In tomaarsen/attention_sinks;

· wln20 opened on Aug 7, 2024

Last generated token getting ignored in streaming.py?

Status: Open.

#45 In tomaarsen/attention_sinks;

· ritik99 opened on Mar 29, 2024

TypeError: 'NoneType' object is not subscriptable

Status: Open.

#43 In tomaarsen/attention_sinks;

· Kuchiriel opened on Jan 16, 2024

Support newer versions of mistral (e.g. mistralai/Mistral-7B-Instruct-v0.2)?

Status: Open.

#41 In tomaarsen/attention_sinks;

· spring1915 opened on Jan 10, 2024

chatglm3 support?

Status: Open.

#40 In tomaarsen/attention_sinks;

· ScottishFold007 opened on Dec 26, 2023

3.3: Learnable Sink Token

Status: Open.

#38 In tomaarsen/attention_sinks;

· photomz opened on Dec 20, 2023

KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

Status: Open.

#37 In tomaarsen/attention_sinks;

· pseudotensor opened on Dec 18, 2023

Error when using Qwen 7b chat

Status: Open.

#36 In tomaarsen/attention_sinks;

· Minami-su opened on Dec 11, 2023

Error loading Qwen-1_8B

Status: Open.

#35 In tomaarsen/attention_sinks;

· haiphong93 opened on Dec 4, 2023

Generation stop；torch.cuda.OutOfMemoryError: CUDA out of memory.

Status: Open.

#34 In tomaarsen/attention_sinks;

· Essence9999 opened on Nov 24, 2023

GPTQ models support

Status: Open.

#31 In tomaarsen/attention_sinks;

· synacktraa opened on Nov 20, 2023