25 results found Sort:

jphall663
awesome-machine-learning-interpretability jphall663
578
3.5k
cc0-1.0
131
A curated list of awesome responsible machine learning resources.
r xai python awesome fairness ai-safety secure-ml reliable-ai awesome-list data-science transparency explainable-ml interpretability interpretable-ai interpretable-ml machine-learning interpretable-machine-learning privacy-enhancing-technologies machine-learning-interpretability privacy-preserving-machine-learning
Created 2018-06-21
1,043 commits to master branch, last one a day ago
Giskard-AI
giskard Giskard-AI
215
3.4k
apache-2.0
27
🐢 Open-Source Evaluation & Testing for LLMs and ML models
llm mlops llmops llm-eval ai-safety ml-safety ai-testing ml-testing ai-red-team ai-security fairness-ai llm-security ml-validation llm-evaluation rag-evaluation red-team-tools responsible-ai trustworthy-ai model-monitoring ethical-artificial-intelligence
Created 2022-03-06
9,675 commits to main branch, last one a day ago
PKU-Alignment
safe-rlhf PKU-Alignment
104
1.2k
apache-2.0
17
Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback
gpt llm llms rlhf llama alpaca beaver safety vicuna datasets ai-safety deepspeed safe-rlhf transformer transformers large-language-models reinforcement-learning safe-reinforcement-learning reinforcement-learning-from-human-feedback safe-reinforcement-learning-from-human-feedback
Created 2023-05-15
110 commits to main branch, last one about a month ago
tigerlab-ai
tiger tigerlab-ai
26
385
apache-2.0
11
Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding, RAG), TigerTune (fine-tuning)
llm rag aisafety ai-safety fine-tuning llm-training classification data-augmentation large-language-models
Created 2023-10-23
137 commits to main branch, last one 6 months ago
agencyenterprise
PromptInject agencyenterprise
27
276
mit
9
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Sa...
agi gpt-3 ai-safety ml-safety ai-alignment agi-alignment language-models chain-of-thought machine-learning prompt-engineering adversarial-attacks large-language-models
Created 2022-10-25
2 commits to main branch, last one about a year ago
ShengranHu
Thought-Cloning ShengranHu
20
233
mit
2
[NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking
pytorch ai-safety deep-learning imitation-learning reinforcement-learning artificial-intelligence
Created 2023-01-13
15 commits to main branch, last one 3 months ago
hendrycks
ethics hendrycks
36
217
mit
7
Aligning AI With Shared Human Values (ICLR 2021)
gpt-3 ai-safety ml-safety ethical-ai machine-ethics
Created 2020-08-06
25 commits to master branch, last one about a year ago
normster
llm_rules normster
14
197
apache-2.0
3
RuLES: a benchmark for evaluating rule-following in language models
gpt-4 ai-safety ai-security
Created 2023-11-03
22 commits to main branch, last one about a month ago
lets-make-safe-ai
make-safe-ai lets-make-safe-ai
7
169
unknown
2
How to Make Safe AI? Let's Discuss! 💡|💬|🙌|📚
ai agi ai-safety ai-alignment artificial-intelligence artificial-general-intelligence
Created 2023-02-27
11 commits to main branch, last one about a year ago
tomekkorbak
pretraining-with-human-feedback tomekkorbak
14
167
mit
6
Code accompanying the paper Pretraining Language Models with Human Preferences
gpt rlhf ai-safety pretraining ai-alignment language-models decision-transformers reinforcement-learning
Created 2023-02-20
5 commits to master branch, last one 3 months ago
Giskard-AI
awesome-ai-safety Giskard-AI
11
140
apache-2.0
3
📚 A curated list of papers & technical articles on AI Quality & Safety
ai ml llm mlops llmops awesome ai-safety ml-safety ai-quality ethical-ai ml-testing robustness ai-alignment awesome-list model-testing computer-vision machine-learning model-validation artificial-intelligence natural-language-processing
Created 2023-04-19
28 commits to main branch, last one about a year ago
WindVChen
DiffAttack WindVChen
10
111
apache-2.0
3
An unrestricted attack based on diffusion models that can achieve both good transferability and imperceptibility.
ai-safety diffusion-models adverarial-attacks transferable-attacks unrestricted-attacks imperceptible-attacks diffusion-adversarial-attack
Created 2023-04-29
11 commits to main branch, last one 3 months ago
ryoungj
ToolEmu ryoungj
7
90
apache-2.0
4
A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use
agent ai-safety language-agent language-model prompt-engineering large-language-models
Created 2023-09-26
13 commits to main branch, last one 2 months ago
microsoft
SafeNLP microsoft
6
90
other
6
Safety Score for Pre-Trained Language Models
nlp ai-safety fairness-ai
Created 2022-07-02
31 commits to main branch, last one 7 months ago
PKU-YuanGroup
Hallucination-Attack PKU-YuanGroup
11
85
mit
2
Attack to induce LLMs within hallucinations
llm nlp ai-safety llm-safety deep-learning hallucinations machine-learning adversarial-attacks
Created 2023-09-29
22 commits to master branch, last one 7 months ago
PKU-Alignment
beavertails PKU-Alignment
3
83
apache-2.0
5
BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).
gpt llm llms rlhf llama beaver safety datasets ai-safety safe-rlhf human-feedback language-model human-feedback-data large-language-model
Created 2023-06-14
3 commits to main branch, last one 10 months ago
megvii-research
FSSD_OoD_Detection megvii-research
12
80
mit
7
Feature Space Singularity for Out-of-Distribution Detection. (SafeAI 2021)
anomaly ai-safety ood-detection anomaly-detection out-of-distribution-detection
Created 2020-08-05
15 commits to master branch, last one 3 years ago
EzgiKorkmaz
adversarial-reinforcement-learning EzgiKorkmaz
5
77
unknown
5
Reading list for adversarial perspective and robustness in deep reinforcement learning.
ai-safety safe-rlhf ai-alignment explainable-rl responsible-ai adversarial-attacks adversarial-policies machine-learning-safety robust-machine-learning deep-reinforcement-learning meta-reinforcement-learning safe-reinforcement-learning adversarial-machine-learning explainable-machine-learning reinforcement-learning-safety robust-reinforcement-learning multiagent-reinforcement-learning adversarial-reinforcement-learning reinforcement-learning-generalization robust-adversarial-reinforcement-learning
Created 2023-09-08
14 commits to main branch, last one 8 months ago
dlmacedo
entropic-out-of-distribution-detection dlmacedo
10
75
apache-2.0
4
A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Perform efficient inferences (i.e., do not increase inference tim...
ood osr pytorch open-set ai-safety deep-learning ood-detection trustworthy-ai machine-learning anomaly-detection novelty-detection out-of-distribution open-set-recognition robust-machine-learning trustworthy-machine-learning out-of-distribution-detection
Created 2019-08-16
50 commits to master branch, last one about a year ago
SafeAILab
RAIN SafeAILab
4
68
bsd-2-clause
1
[ICLR'24] RAIN: Your Language Models Can Align Themselves without Finetuning
ai-safety alignment large-language-models
Created 2023-10-08
31 commits to main branch, last one 9 days ago
ai4ce
FLAT ai4ce
10
66
apache-2.0
5
[ICCV2021 Oral] Fooling LiDAR by Attacking GPS Trajectory
gnss lidar robotics ai-safety point-cloud 3d-perception deep-learning trustworthy-ai autonomous-driving 3d-object-detection adversarial-attacks trustworthy-machine-learning
Created 2020-10-06
27 commits to master branch, last one about a year ago
dit7ya
awesome-ai-alignment dit7ya
10
57
unknown
5
A curated list of awesome resources for getting-started-with and staying-in-touch-with Artificial Intelligence Alignment research.
awesome ai-safety ai-alignment awesome-list
Created 2018-11-16
19 commits to master branch, last one 10 months ago
dlmacedo
distinction-maximization-loss dlmacedo
5
45
apache-2.0
3
A project to improve out-of-distribution detection (open set recognition) and uncertainty estimation by changing a few lines of code in your project! Perform efficient inferences (i.e., do not increas...
ood osr pytorch open-set ai-safety deep-learning ood-detection classification trustworthy-ai machine-learning anomaly-detection novelty-detection out-of-distribution open-set-recognition uncertainty-estimation robust-machine-learning trustworthy-machine-learning out-of-distribution-detection
Created 2022-05-10
40 commits to master branch, last one about a year ago
wesg52
sparse-probing-paper wesg52
10
39
mit
2
Sparse probing paper full code.
ai-safety ai-alignment interpretability mechanistic-interpretability
Created 2023-05-02
5 commits to main branch, last one 5 months ago
StampyAI
stampy-ui StampyAI
9
31
mit
7
AI Safety Q&A web frontend
ai-safety
Created 2022-02-17
1,059 commits to master branch, last one 6 days ago