multihead_att {quickcode} | R Documentation |
Multi-Head Attention
Description
This function implements a simplified version of multi-head attention for sequence data.
Usage
multihead_att(query, key, value, num_heads)
Arguments
query |
A numeric matrix representing the query. |
key |
A numeric matrix representing the key. |
value |
A numeric matrix representing the value. |
num_heads |
An integer specifying the number of attention heads. |
Value
A numeric matrix representing the attention output.
Examples
query <- matrix(rnorm(12), nrow = 3)
key <- matrix(rnorm(12), nrow = 3)
value <- matrix(rnorm(12), nrow = 3)
multihead_att(query, key, value, num_heads = 2)
[Package quickcode version 1.0.8 Index]