15 results found Sort:

imperial-qore
TranAD imperial-qore
169
560
bsd-3-clause
10
[VLDB'22] Anomaly Detection using Transformers, self-conditioning and adversarial training.
anomaly-detection transformer-models adversarial-learning multi-head-attention unsupervised-learning
Created 2021-03-01
110 commits to main branch, last one about a year ago
therealoliver
Deepdive-llama3-from-scratch therealoliver
41
557
mit
4
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.
gpt rms llms mask rope llama swiglu kv-cache rms-norm attention inference residuals tokenizer transformer language-model llm-configuration attention-mechanism positional-encoding multi-head-attention rotary-position-encoding
Created 2025-02-19
9 commits to main branch, last one 26 days ago
sooftware
attentions sooftware
70
528
mit
3
PyTorch implementation of some attentions for Deep Learning Researchers.
pytorch attention additive-attention multi-head-attention dot-product-attention location-aware-attention location-sensitive-attension relative-positional-encoding relative-multi-head-attention
Created 2020-03-21
89 commits to master branch, last one 3 years ago
anicolson
DeepXi anicolson
127
507
mpl-2.0
25
Deep Xi: A deep learning approach to a priori SNR estimation implemented in TensorFlow 2/Keras. For speech enhancement and robust ASR.
tcn mmse keras deepxi mhanet resnet deep-xi deepmmse mmse-lsa attention robust-asr tensorflow noise-estimation residual-networks source-separation speech-separation speech-enhancement multi-head-attention a-priori-snr-estimator minimum-mean-square-error
Created 2018-05-25
425 commits to master branch, last one 3 years ago
poloclub
dodrio poloclub
35
359
mit
5
Exploring attention weights in transformer-based models with linguistic knowledge.
nlp transformer deep-learning visualization attention-mechanism multi-head-attention interactive-visualizations
Created 2020-10-30
186 commits to master branch, last one about a year ago
Rintarooo
VRP_DRL_MHA Rintarooo
38
185
mit
2
"Attention, Learn to Solve Routing Problems!"[Kool+, 2019], Capacitated Vehicle Routing Problem solver
vrp pytorch reinforce tensorflow policy-gradient multi-head-attention deep-reinforcement-learning capacitated-vehicle-routing-problem
Created 2020-06-24
89 commits to master branch, last one 4 years ago
monk1337
Various-Attention-mechanisms monk1337
25
125
unknown
5
This repository contain various types of attention mechanism like Bahdanau , Soft attention , Additive Attention , Hierarchical Attention etc in Pytorch, Tensorflow, Keras
keras pytorch attention attention-lstm self-attention attention-model luong-attention attention-network bahdanau-attention sentence-attention attention-mechanism attention-mechanisms multi-head-attention hierarchical-attention scaled-dot-product-attention
Created 2018-07-04
59 commits to master branch, last one 3 years ago
datnnt1997
multi-head_self-attention datnnt1997
16
72
unknown
1
A Faster Pytorch Implementation of Multi-Head Self-Attention
attention multi-head self-attention attention-mechanism multihead-attention multi-head-attention transformer-attention pytorch-self-attention multihead-self-attention multi-head-self-attention
Created 2020-07-28
6 commits to master branch, last one 2 years ago
youngbin-ro
Multi2OIE youngbin-ro
18
56
mit
4
Multi^2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT (Findings of ACL: EMNLP 2020)
bert multilingualism sequence-labeling multi-head-attention information-extraction open-information-extraction
Created 2020-09-17
24 commits to master branch, last one 2 years ago
JacobHanimann
scDINO JacobHanimann
10
49
apache-2.0
4
Self-Supervised Vision Transformers for multiplexed imaging datasets
morphology phenotyping single-cell unsupervised multi-channel greyscale-image vision-transformer multi-head-attention high-content-screening self-supervised-learning fluorescence-microscopy-imaging
Created 2023-01-16
33 commits to master branch, last one 8 months ago
knotgrass
attention knotgrass
10
47
unknown
2
several types of attention modules written in PyTorch for learning purposes
pytorch attention transformer transformers softmax-layer attention-mechanism multi-head-attention multi-query-attention grouped-query-attention scale-dot-product-attention
Created 2023-06-28
45 commits to main branch, last one 5 months ago
engelnico
point-transformer engelnico
8
40
unknown
2
This is the official repository of the original Point Transformer architecture.
pytorch sortnet modelnet modelnet40 transformer segmentation shapenetpart deep-learning 3d-point-cloud 3d-pointclouds classification 3d-segmentation modelnet-dataset shapenet-dataset 3d-classification attention-mechanism multi-head-attention transformer-architecture
Created 2021-08-24
9 commits to main branch, last one 2 years ago
IParraMartin
An-Explanation-Is-All-You-Need IParraMartin
1
36
unknown
2
The original transformer implementation from scratch. It contains informative comments on each block
ai gpt nlp gpt-2 gpt-3 gpt-4 pytorch begginers transformer translation deep-learning language-model machine-learning begginer-friendly attention-mechanism multi-head-attention artificial-intelligence
Created 2024-06-15
41 commits to main branch, last one 9 months ago
Bruce-Lee-LY
flash_attention_inference Bruce-Lee-LY
3
35
bsd-3-clause
1
Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
gpu llm mha cuda nvidia cutlass inference tensor-core flash-attention flash-attention-2 large-language-model multi-head-attention
Created 2023-08-16
1 commits to master branch, last one 23 days ago
Bruce-Lee-LY
decoding_attention Bruce-Lee-LY
2
35
bsd-3-clause
2
Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.
gpu gqa llm mha mla mqa cuda nvidia flashmla cuda-core inference flashinfer flash-attention decoding-attention large-language-model multi-head-attention
Created 2024-08-14
2 commits to master branch, last one 13 days ago