DCT-former: efficient self-attention with discrete cosine transform

DOI10.1007/S10915-023-02125-5arXiv2203.01178OpenAlexW4319350749MaRDI QIDQ6158987

Marco Prato, Carmelo Scribano, Giorgia Franchini, Marko Bertogna

Publication date: 20 June 2023

Published in: Journal of Scientific Computing (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2203.01178

zbMATH Keywords

natural language processingdiscrete cosine transformdeep learningtransformersself-attentionfrequencies domain

Mathematics Subject Classification ID

Statistics (62-XX) Multivariate analysis (62Hxx) Computer science (68-XX)

Cites Work

• Unnamed Item
• Novel design of multiplier-less FFT processors
• Type-II/III DCT/DST algorithms with reduced number of arithmetic operations
• A fast cosine transform in one and two dimensions
• Learning Fast Sparsifying Transforms
• Discrete Cosine Transform

This page was built for publication: DCT-former: efficient self-attention with discrete cosine transform