multihead_att {quickcode}R Documentation

Multi-Head Attention

Description

This function implements a simplified version of multi-head attention for sequence data.

Usage

multihead_att(query, key, value, num_heads)

Arguments

query

A numeric matrix representing the query.

key

A numeric matrix representing the key.

value

A numeric matrix representing the value.

num_heads

An integer specifying the number of attention heads.

Value

A numeric matrix representing the attention output.

Examples

query <- matrix(rnorm(12), nrow = 3)
key <- matrix(rnorm(12), nrow = 3)
value <- matrix(rnorm(12), nrow = 3)
multihead_att(query, key, value, num_heads = 2)


[Package quickcode version 1.0.8 Index]