NLP

Can LLM Have Spatial Intelligence?

Recently, our school celebrated the 60th anniversary of Computer Science & AI. To mark the occasion, the organizers invited Fernando Pereira to deliver a lecture on the connection between form and meaning in language. This subject has captivated the minds of linguists, computer scientists, and cognitive researchers for many years. During the lecture, Pereira presented a thought-provoking example. He posed the question: “Bob is sitting to the right of Alice, and Jack is to the left of Bob.

October 13, 2023 Read

Contrastive Learning Note

Related reading: The Beginner’s Guide to Contrastive Learning SimCSE: Simple Contrastive Learning of Sentence Embeddings A Simple Framework for Contrastive Learning of Visual Representations Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Background Contrastive learning aims to learn effective representation by pulling semantically close neighbors together and pushing apart non-neighbors. Initially, contrastive learning was applied to computer vision tasks. As what it is shown in the figure below, we expect the model to learn the communities between two images that share the same label and the difference between a pair of images with different labels.

August 8, 2022 Read

Contrastive Learning Note

Related reading: The Beginner’s Guide to Contrastive Learning SimCSE: Simple Contrastive Learning of Sentence Embeddings A Simple Framework for Contrastive Learning of Visual Representations Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Background Contrastive learning aims to learn effective representation by pulling semantically close neighbors together and pushing apart non-neighbors. Initially, contrastive learning was applied to computer vision tasks. As what it is shown in the figure below, we expect the model to learn the communities between two images that share the same label and the difference between a pair of images with different labels.

August 8, 2022 Read

16-Semantic Role Labelling and Argument Structure

1. Semantic Roles 1.1 Commonly used thematic roles 1.2 Issues with thematic roles No universally agreed-upon set of roles Items with the same role may not behave quite the same Problem avoid: Specific to individual verbs only (PropBank) Specific to small groups of verbs (FrameNet) #2. Labelling 2.1 PropBank Frame Roles: Arg0-PAG: loader Arg1-GOL: beast of burden Arg2-PPT: cargo Arg3-MNR: instrument 2.2 FrameNet ==Meanings are reletavised to scenes!== Tries to capture relationships among word and phrase meanings by assigning them the same frame $\approx 1000$ frames represent scenarios Frames are explained with textual descriptions and linguistic examples Resources:

October 14, 2020 Read

15-Distributional Semantics

Distributional hypothesis: Can infer meaning just by looking at the contexts a word occurs in Meaning IS the contexts a word occurs in either way , similar contexts imply similar meanings 1. Distributional semantics 1.1 Basic idea Represent each word $w_i$ as a vector of its contexts Distributional semantic models also called vector-space models 1.2 Context 1.2.1 Defining the context usually ignore stopwords usually use a large window around the target word 1.

October 13, 2020 Read

14-Lexical Semantics

Goal: a machine which can answer questions in natural language can access to KB can access to vast quantities of English text 1. Semantics 1.1 Lexical and Sentential Lexical semantics: the meaning of individual words Sentential semantics: how word meanings combine (e.g. who did that to whom; when, how, why…) 1.2 Problems Senses: words may have different meanings (need to disambiguate between them) Synonyms: words may have the same meaning Ontology (hyponym & hypernym): words can refer to a subset or superset of the concept referred to by another word (A is a B relationships) Similarity & gradation: words may be related in other ways Inference PS: some of these problems can be solved with a good ontology (e.

October 12, 2020 Read

13-Heads, Dependency parsing

1. Parser evaluation Correct: if there is a gold constituent that spans the same sentence positions Hasher measure: also require the constituent labels to match precision recall F-score (balances precision and recall): $$ \frac{2pr}{p+r} $$ 2. Lexicalisation 2.1 Introduction Problem: replacing one word with another with the same POS will never result in a different parsing decision, even though it should Solution: adding the lexical head of the phrase 2.2 Pros & Cons Pros:

October 11, 2020 Read

11,12-Syntax and Parsing

Two theories of syntax: Context-free grammar Dependency grammar 1. Context-free grammer Symbols: Terminals (t): words Non-terminals (NT): phrasal categories (e.g. S, NP, VP, PP) with S being the start symbol Chomsky Normal Form: only has rules of the form $$ NT \rightarrow NT \ \ NT \ \ \ NT\rightarrow t $$ 2. Syntactic analysis 2.1 Global Ambibuity Structural ambiguity POS ambiguity: Attachment ambiguity: Some sentences have structural ambiguity even without part-of-speech ambiguity

May 11, 2020 Read

10-Annotation & Evaluation

1. Annotation Penn Treebank: 36 POS tags (excluding punctuation) Factors in Annotation: source data (genre? size? licensing?) annotation scheme (complexity? guidelines?) annotators (expertise? training?) annotation software (graphical interface?) quality control procedures (multiple annotation, adjudication?) 2. Evaluation 2.1 Hypotheses About existing linguistic objects: Is this text by Shakespeare or Marlowe? About output of a language system: How well does this language model predict the data? How accurate is this segmenter / tagger / parser?

March 26, 2020 Read

Real-Time Speech Recognition Using Python

A step-by-step guidance to create a simple real-time speech recogniser

March 5, 2020 Read

8,9-POS tagging and HMMs

Use Hidden Markov Models to do POS tagging Notation: Sequence of observation overtime (sentence): $ O=o_1\dots o_T $ Sequence of states (tags): $Q=q^1\dots q^N$ Sequence states over time: $Q = q_1 \dots q_T$ Vocabulary size: $V$ Time step: $t$, not a tag Matrix of transition probabilities: $A$ $a_{ij}$: the prob of transitioning from state $q^i$ to $q^j$ Matrix of output probabilities: $B$ $b_i(o_t)$: the prob of emitting $o_t$ from state $q^i$ 1.

February 11, 2020 Read

7-Text Classification

Introduce two different methods to do text classification: Naive Bayes and Maximum Entropy 1. Introduction Application: spam detection (binary: spam / not spam) sentiment analysis (binary or multiway) review (pos / neg or 1-5 stars) political argument (pro / con, or pro / con / neutral) topic classification (multiway: sport / finance / travel, etc.) categorise the author native language identification diagnosis of disease gender, dialect, educational background, political, orientation Why not N-gram:

February 4, 2020 Read