Skip to content

Commit f179ef7

Browse files
committed
Update dependencyvit.py
1 parent a067d19 commit f179ef7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

timm/models/dependencyvit.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ def forward(self, in_tuple: Tuple[torch.Tensor, torch.Tensor]) -> Tuple[torch.Te
7878
p = (self.head_selector(x) / self.head_selector_temperature).softmax(dim=-1)
7979
p = p.transpose(-2, -1).reshape(B, self.num_heads, 1, N)
8080

81-
m = self.message_controller(x).sigmoid().reshape(B, 1, 1, N)# * m
81+
m = self.message_controller(x).sigmoid().reshape(B, 1, 1, N) * m
8282

8383
q = q * self.scale
8484
attn = q @ k.transpose(-2, -1)

0 commit comments

Comments
 (0)