LIN 404 — Computational Linguistics II: Lecture Note

About this lecture note

This is the complete lecture note for LIN 404 Computational Linguistics II. You can read it online below, open it in the full viewer, or download the PDF for offline study. It pairs with the course's video lectures, so you can read and watch together.

Preview: opening of the note

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS Course code: LIN 404 Course title: Computational Linguistics II Course credit load: 2 Units Series 1: Natural Language Processing Problems and Corpus Methods Series 2: Probability, N-Grams, and Language Models Series 3: Computational Phonetics, Morphology, and Syntax Series 4: Computational Semantics and Meaning Representation Series 5: Applications and Performance Evaluation in Computational Linguistics Proposed Outline Series for 1 Part 1.1: Natural language processing problems and perspectives • Sub-Part 1.1.1: Key challenges in NLP • Sub-Part 1.1.2: Perspectives on solving NLP problems Part 1.2: Corpus linguistics foundations • Sub-Part 1.2.1: Definition and role of corpora • Sub-Part 1.2.2: Methods of corpus construction and annotation • Sub-Part 1.2.3: Applications of corpus linguistics in NLP Part 1.3: Linking NLP problems with corpus methods • Sub-Part 1.3.1: Using corpora to address NLP challenges • Sub-Part 1.3.2: Case examples of corpus-based NLP solutions Introduction Natural language processing (NLP) is at the heart of modern computational linguistics, powering applications such as machine translation, speech recognition, and text analysis. Yet, working with human language presents unique challenges, from ambiguity and variability to the sheer scale of linguistic data. To address these challenges, corpus linguistics provides structured collections of texts that serve as the foundation for empirical analysis. This Series introduces you to the problems and perspectives in NLP and demonstrates how corpus methods can be applied to tackle them effectively. The aim of this Series is to familiarise you with the key challenges in natural language processing and to equip you with the knowledge of corpus linguistics as a methodological tool for addressing these challenges.

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS We shall commence this Series by examining natural language processing problems and perspectives, highlighting the complexities of human language and the approaches used to manage them. Next, we will explore the foundations of corpus linguistics, including the definition, construction, and annotation of corpora, as well as their applications in computational linguistics. Finally, we will bring the Series to a close by linking NLP problems with corpus methods, showing how corpora can be strategically employed to address real-world language processing challenges through case examples. Series Learning Outcomes Upon completing this Series, you should be able to: • SLO 1.1 identify and explain key challenges in natural language processing problems and perspectives, • SLO 1.2 describe the role and definition of corpora in computational linguistics, • SLO 1.3 demonstrate how corpora are constructed and annotated for linguistic analysis, • SLO 1.4 analyse applications of corpus linguistics in addressing NLP challenges, and • SLO 1.5 apply corpus-based methods to illustrate solutions to selected NLP problems. 1.1 Natural Language Processing Problems And Perspectives 1.1.1 Key challenges in NLP Natural language processing (NLP) refers to the computational study of human language, aiming to enable machines to analyse, interpret, and generate text or speech. One of the main challenges in NLP is ambiguity, which occurs when a word, phrase, or sentence has multiple possible meanings. For example, the word bank can mean a financial institution or the side of a river. Another challenge is variability, defined as the wide range of ways people express the same idea, such as I am happy versus I feel joyful. These challenges make it difficult for computational systems to achieve accurate interpretation. Fill in the blank: The word bank can refer to a financial institution or the side of a ______.

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS Figure 1.1: Key challenges in natural language processing 1.1.2 Perspectives on solving NLP problems Different perspectives have been developed to address NLP challenges. The rule-based perspective relies on manually crafted grammatical rules to process language. While precise, it struggles with scalability and flexibility. The statistical perspective uses probability and large datasets to predict linguistic patterns, making it more adaptable to variation. More recently, the machine learning perspective has become dominant, applying algorithms that learn from data to improve performance over time. Each perspective contributes unique strengths, and modern NLP often combines them for better results. Fill in the blank: The statistical perspective in NLP relies on ______ and large datasets to predict linguistic patterns. Perspective Methodology Primary Advantage Rule-Based Relies on explicit, hand-coded grammatical rules, lexicons, and "if-then" logic. High Interpretability: You know exactly why the system made a specific decision. Statistical Uses probability distributions and large datasets to predict the most likely next word or structure (e.g., Hidden Markov Models). Robustness: Handles variation and "noisy" data better than strict rules. Machine Learning Applies algorithms (and increasingly deep learning/neural networks) that automatically learn features and patterns from data. Scalability: Capable of capturing complex, non-linear relationships in language without manual feature engineering. Box 1.1: Perspectives on solving NLP problems Pilot questions 1.1 1. What does NLP stand for? 2. What is ambiguity in NLP, and give one example. 3. What is variability in NLP, and give one example. 4. What does the rule-based perspective rely on? 5. How does the statistical perspective address NLP problems? 6. What is the main advantage of the machine learning perspective?

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS 1.2 Corpus Linguistics Foundations 1.2.1 Definition and role of corpora A corpus is a large, structured collection of texts or speech samples used for linguistic analysis. In computational linguistics, corpora serve as the foundation for empirical research, allowing researchers to observe language patterns across authentic data rather than relying solely on intuition. Corpora are essential for training natural language processing systems, as they provide the raw material for tasks such as part-of-speech tagging, parsing, and semantic analysis. Fill in the blank: A corpus is a structured collection of ______ used for linguistic analysis. Figure 1.2: The role of corpora in computational linguistics 1.2.2 Methods of corpus construction and annotation Building a corpus involves several steps, including text collection, cleaning, and annotation. Annotation refers to the process of adding linguistic information to raw text, such as part-of-speech tags, syntactic structures, or semantic roles. Annotated corpora are particularly valuable because they provide structured data for training and evaluating NLP models. For example, the Penn Treebank is a widely used annotated corpus that includes syntactic trees for English sentences. Fill in the blank: Adding linguistic information such as part-of-speech tags to raw text is called ______. Phase Description Key Activities 1. Text Collection Gathering the raw data based on defined criteria (genre, time period, region). Web scraping, digitizing print media, or recording spoken language.

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS Phase Description Key Activities 2. Cleaning & Formatting Removing "noise" and standardizing the data into a machine-readable format. Strip HTML tags, fix encoding errors, and normalize whitespace. 3. Annotation Adding linguistic "tags" or labels to the text to make the data more useful. Part-of-Speech (POS) tagging, Lemmatization, and Syntactic Treebanking. 4. Validation & QC Checking the accuracy and consistency of the annotations and data. Inter-annotator agreement (IAA) checks and automated error detection. Box 1.2: Steps in corpus construction 1.2.3 Applications of corpus linguistics in NLP Corpus linguistics provides practical applications in natural language processing. For instance, corpora are used to train language models, which predict word sequences based on probability. They also support lexical analysis, helping identify word frequencies and collocations. In addition, corpora are crucial for evaluation, as they provide benchmark datasets against which NLP systems can be tested. By grounding NLP in real-world data, corpus linguistics ensures that computational models reflect authentic language use. Fill in the blank: Corpora are essential for training ______ models, which predict word sequences based on probability. Figure 1.3: Applications of corpus linguistics in NLP Pilot questions 1.2 1. What is a corpus, and why is it important in computational linguistics? 2. What is annotation in corpus construction?

LA-NET | ONLINE LEARNING VIDEO PLATFORM FOR POST-SECONDARY & HIGHER INSTITUTION STUDENTS 3. Name one widely used annotated corpus. 4. List two steps involved in corpus construction. 5. What are language models trained on? 6. How do corpora support evaluation in NLP? 1.3 Linking NLP Problems With Corpus Methods 1.3.1 Using corpora to address NLP challenges One of the most effective ways to tackle natural language processing challenges is through the use of corpora, which provide authentic language data for analysis. For example, ambiguity in words such as bank can be resolved by examining how the word is used across thousands of sentences in a corpus. Similarly, variability in expressions can be managed by identifying common collocations and patterns. Corpora allow computational models to learn from real-world usage rather than relying solely on theoretical rules. Fill in the blank: Ambiguity in words such as bank can be resolved by examining how the word is used across thousands of sentences in a ______. Figure 1.4: Using corpora to address NLP challenges 1.3.2 Case examples of corpus-based NLP solutions Several case studies demonstrate how corpus methods directly solve NLP problems.

This is the opening of the note. Read the full note or download the PDF to continue.