Posts

My Superficial Views on the Future Development of AI

Today, I happened to discuss the future development trends of AI with some friends. Regarding the future of AI in the coming years, the core viewpoint is that people might be “overly optimistic” about AI development, and the current estimation of AI’s capabilities is overestimated. Without “killer” application scenarios, is it possible that the AI boom could subside in two to three years, eventually bursting the “huge bubble”? I also believe that the actual performance of AI today is far below the public’s expectations.

August 2, 2024 Read

Can LLM Have Spatial Intelligence?

Recently, our school celebrated the 60th anniversary of Computer Science & AI. To mark the occasion, the organizers invited Fernando Pereira to deliver a lecture on the connection between form and meaning in language. This subject has captivated the minds of linguists, computer scientists, and cognitive researchers for many years. During the lecture, Pereira presented a thought-provoking example. He posed the question: “Bob is sitting to the right of Alice, and Jack is to the left of Bob.

October 13, 2023 Read

EACL 2023 Trip to Croatia

I went to Dubrovnik, Croatia for the EACL 2023 conference. It was my first time there. I made new friends and had an amazing time watching the most beautiful sunset in Kings Landing.

May 11, 2023 Read

Contrastive Learning Note

Related reading: The Beginner’s Guide to Contrastive Learning SimCSE: Simple Contrastive Learning of Sentence Embeddings A Simple Framework for Contrastive Learning of Visual Representations Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Background Contrastive learning aims to learn effective representation by pulling semantically close neighbors together and pushing apart non-neighbors. Initially, contrastive learning was applied to computer vision tasks. As what it is shown in the figure below, we expect the model to learn the communities between two images that share the same label and the difference between a pair of images with different labels.

August 8, 2022 Read

Contrastive Learning Note

Related reading: The Beginner’s Guide to Contrastive Learning SimCSE: Simple Contrastive Learning of Sentence Embeddings A Simple Framework for Contrastive Learning of Visual Representations Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Background Contrastive learning aims to learn effective representation by pulling semantically close neighbors together and pushing apart non-neighbors. Initially, contrastive learning was applied to computer vision tasks. As what it is shown in the figure below, we expect the model to learn the communities between two images that share the same label and the difference between a pair of images with different labels.

August 8, 2022 Read

Transformer - Attention Is All You Need

Brief introduction to the famous paper about Transformers: Attention Is All You Need Transformer Paper: Attention Is All You Need Github link of Transformer 1. Background encoder-decoder self attention feed-forward network positional encoding Dataset: WMT 2014 English-to-French translation task Evaluation: BLEU (bilingual evaluation understudy) BLEU computes the modified precision metric using n-grams to measure the similarity between the candidate text and reference text. The idea is, if a word in reference is already matched, this word cannot be matched again.

January 8, 2021 Read

Singular Value Decomposition

Brief introduction to Singular Value Decomposition (SVD) 1. What is SVD (Singular Value Decomposition)? 1.1 Definition Define $ \mathbf{ X } = (\vec{x_1}) $ as a $n\times m$ matrix ($n \ge m$). A fraction of $\mathbf{X}$ is: $$ \mathbf{X} = \mathbf{ U }\mathbf{ \Sigma } \mathbf{ V }^\top $$ where: $ \mathbf{U} $: $n \times n$ unitary matrix ($\mathbf{U}^* = \mathbf{U}^{-1}$) $ \mathbf{\Sigma} $: $n \times m$ rectangular diagonal matrix with non-negative real numbers on the diagonal

October 27, 2020 Read

Real-Time Speech Recognition Using Python

A step-by-step guidance to create a simple real-time speech recogniser

March 5, 2020 Read

Interesting exploration of i++ and ++i in Java

An interesting discover of using i++ and ++i in Java loop

February 24, 2020 Read

Structural Testing

Def: judging test suite thoroughness based on the structure of the program itself Also known as white-box testing Distinguish from functional testing (black-box testing) structural testing is still testing product functionality against its specification only the measure of thoroughness has changed Why need structural testing: Executing all control flow elements does not guarantee finding all faults Increase confidence in thoroughness of testing Practical use: Firstly create functional test suite Then measure structural coverage to identify see what is missing Statement testing & Branch testing Statement testing:

February 5, 2020 Read