Liwen's Notes

Python Programming Language

Tue, 09 Sep 2025 00:00:00 GMT

Python is a scripting language that runs on an interpreter.

Under the hood, Python scripts are compiled into byte code and run by the Python Virtual Machine (PVM) - the runtime engine that loops through byte code instructions individually.

Python supports procedural, OOP and functional programming paradigms with classes, lambdas, generators, decorators, first-class objects and comprehension expressions.

Mutable vs Immutable Objects in Python

Everything in Python is an object, and every object can be classified as mutable or immutable. Numbers, strings and tuples are immutable, while lists, dictionaries and sets are not. As Python only changes references when updating content for mutable types (similar to pointers in C), changing a referenced object in place may impact multiple references from within different objects.

Different mutable types have different built-in methods to make copies. For example, the slicing or range operations of lists yield new copies, and dictionaries have a copy() method. All the copies made in this way are shallow, meaning that only the top-level references are copied. To make a deep copy of a nested object recursively, use the deepcopy() from the copy module.

Dynamic Typing in Python

Python is dynamically and strongly typed. There are no type declarations in Python. Instead, the syntax of code expressions determines the types of objects. For example, square brackets represent lists, and curly braces define dictionaries. A variable is created when you assign it a value, and a variable must be assigned a value before you can use it.

In Python, the type information is associated with objects, not variables. Each Python object contains a header field that tags the object with a type (implemented as a pointer to the type object). For example, expression a = 'Python' creates a ‘typeless’ variable a that points to an object. The object has a type tag pointing to the Python object str, which is the type of the object.

Garbage Collection & References

Certain frequently used immutable objects, such as the number 1 or the string a, may not be reclaimed by the garbage collector immediately after they lose all the references from user-defined variables (the reference count drops to 0). The Python runtime may cache them, keep the memory around, and reuse the objects later. In addition, the Python runtime often references certain objects and keeps them in memory. For example, the number 1 will certainly have a larger reference count than one may expected.

import sys
sys.getrefcount(1)
 
# 5854

Data Types

In Python, types in the same category share the same set of operations. For example, all sequence types, such as strings, lists and tuples, support indexing, slicing and concatenation operations. Strings are also immutable and, like other immutable types (frozen sets and tuples), do not support in-place changes.

There are three main data type categories in Python.

Number

Number types include integer, floating point, decimal, fraction and others.

Sequence

Sequence types include string, list, tuple, range, collection.deque, array, and memeryview, and more. They support iteration, unpacking, indexing, slicing, and concatenation operations. Apart from types with atomic values such as string, sequences also support pattern matching.

In Python, there are two types of sequences: flat sequences and container sequences. Container sequences contain references to other objects, while flat sequences contain atomic values. The two have very different memory models. Sequence types can also be grouped by mutability. Immutable ones include tuple, str and bytes. With these two types of categorisation, a list can be described as a mutable container sequence.

Core sequence types are implemented in C with performance optimisation built-in.

Slicing

Sequence slicing assignments can be considered a combination of two steps: deletion and insertion of a section of the original object. The length of the deletion and insertion does not have to be the same; hence, it can be used to replace, expand or shrink the object at hand in place using an assignment statement. For example:

L = ['eggs', 'bacon', 'spam']
L[0:2] = ['eat', 'more']
 
L 
# ['eat', 'more', 'spam']
 
L[0:1] = ['I', 'would', 'like', 'to']
L
# ['I', 'would', 'like', 'to', 'eat', 'more', 'spam']
 
 
# When the target of the assignment is a slice, the righthand side MUST be an iterable object, even if it has just one item
L[1:2] = ['want']
# ['I', 'want', 'to', 'eat', 'more', 'spam']

The expression start:stop:step within [] produces a slice(start, stop, step) object. Python calls seq.__getitem__(slice(start, stop, step)) internally. The [] operator can take multiple indexes or slices separated by commas, as well as the eclipse (...) object.

SKU = slice(0, 6)
DESCRIPTION = slice(21, none)
 
for item in lines:
    print(item[SKU], item[DESCRIPTION])

Mapping

Mappings, such as dictionaries, support indexing by key.

Built-in Data Types

Boolean

Like in many scripting languages, although represented by the dedicated notation True and False and printed as the words True and False, internally, the Boolean type is a subtype of int and has the values 0 and 1. You can even use semantically illogical code like True + 4.

All objects in Python have an intrinsic, inherited True or False chracteristic. For example, 0 is false, but other numbers are true; the empty string '' is false, but the string 'Python' is true. This is similar to the truthy property of JavaScript objects.

Numbers

Python supports a comprehensive range of number types, including integers, floating-point numbers, complex numbers, decimals (the Decimal type), rationals (the Fraction type), and sets. The Python ecosystem also offers a wealth of libraries and packages for advanced mathematical and scientific computation, such as matrix and vector processing and sophisticated graphics and plotting.

String

The String type in Python can be seen as a sequence of one-character strings. Like all other sequence types, it supports positional ordering operations such as len() and the indexing expression s[i]. The String type has a rich set of type-specific methods such as splitlines, encode, endwith and isalpha. The complete list can be viewed by dir(str). The String type also supports + (concatenation) and * (repeat) operations through operator overloading (a form of polymorphism, meaning the same operators will behave differently depending on the objects being operated on).

String Interning

Interning allows Python to store only one copy of each distinct string value in memory regardless of how many times it is referenced.

You can use sys.intern()to create interned strings.

Python 3 Documentation

sys.intern() enters string in the table of “interned” strings and return the interned string – which is string itself or a copy. Interning strings is useful to gain a little performance on dictionary lookup – if the keys in a dictionary are interned, and the lookup key is interned, the key comparisons (after hashing) can be done by a pointer compare instead of a string compare. Normally, the names used in Python programs are automatically interned, and the dictionaries used to hold module, class or instance attributes have interned keys.

Interned strings are not immortal; you must keep a reference to the return value of intern() around to benefit from it.

List

Python Lists are positionally ordered collections of objects of arbitrary types.

Lists are of sequence type in Python. Unlike strings, which are also sequences, lists are mutable, providing a flexible data structure for any arbitrary collection, such as files in a directory or to-do items in a task list.

Lists are analogous to the array type in other programming languages, with the difference that Python lists can hold content of arbitrary types. Similar to multidimensional arrays, Python Lists allow arbitrary nesting.

Many built-in Python List methods can modify their content in place, extend or shrink the list (via pop, insert or extend, etc.), or change the entire list (such as sort). However, unlike in C, growing lists by assigning items to indexes that are out of bounds is not permitted in Python, thanks to a feature called bounds checking.

Lists support list comprehension, building new lists by iterating iterable objects without altering the existing source objects. For example:

[row[1] for row in M if row[1] % 2 == 0]

to filter out the old items in column 2 of a matrix or

[M[i][i] for i in [0, 1, 2]

to collect a diagonal from a 3x3 matrix.

Tuple

Tuples are ordered, immutable collections of arbitrary objects. A tuple is a sequence type. Tuple literals are coded in parentheses (as opposed to square brackets for Lists). Compared to lists, tuples use less memory, their lengths never change, and Python has internal optimisations for tuples.

Tuples can be used as:

immutable lists
records with no field names

Tuple supports arbitrary types, arbitrary nesting and many other sequence operations.

t1 = (1, 2, 2, 3, 4)
 
# The parentheses can be omitted when creating a tuple (when the context is not ambiguous to do so)
t2 = 'python', 3.1415926, [1, 2, 3, 5, 8]

Note: As with all immutable collections in Python, tuples store the references to objects and not the objects themselves, which means the referenced objects can still be altered.

t = (1, 2, [3, 4])
t
# (1, 2, [3, 4])
 
t[2][1] = 5
t
# (1, 2, [3, 5])

The recommendation is to avoid putting mutable items in tuples. Considering the following example:

t = (1, 2, [3, 4])
t[2] += [5, 6]
# TypeError: 'tuple' object does not support item assignment
 
print(t)
# (1, 2, [30, 40, 50, 60]

This example illustrates another important fact: augmented assignments (for example, += calls the in-place add function __iadd__ and *= calls __imul__ if implemented) are not atomic operations.

Dictionary

Quote

Python is basically dicts wrapped in loads of syntactic sugar.

‒ Lalo Martins, early digital nomad and Pythonista We

The dict type is fundamental to Python’s implementation. Class and instance attributes, module namespaces, and function keyword arguments are among some of the core Python constructs represented by dictionaries in memory. The _builtins_._dict_ stores all built-in types, objects, and functions.

The Dictionary type is the only built-in mapping type in Python.

Dictionaries are unordered collections of arbitrary types stored and retrieved by keys instead of positional offsets like the sequence types. The literal syntax for dictionaries is curly brackets {key1: value1, key2: value2, ...}. Like lists, dictionaries are mutable and arbitrarily nestable.

Python dictionaries are similar to associative arrays or hashes in other programming languages. Internally, Python dictionaries are implemented as hash tables optimised for speed and efficiency. That means any hashable object can be used as dictionary keys.

Dictionary type does not raise an out-of-range/bound error when accessing (with the dict.get() method; otherwise, it will yield a KeyError) or setting values for non-existing keys, making it ideal for sparse data structures. It can even be used to simulate a ‘flexible list’ when integers are used as keys. Another common use case is to use tuples as keys to represent sparse matrices.

movies = { 1975: 'Holy Grail', 
          1979: 'Life of Brian', 
          1983: 'The Meaning of Life'}
movies[1979]
 
# Output: 'Life of Brian'
 
matrix = {}
matrix[(2, 3, 4)] = 88
matrix[(7, 8, 9)] = 99
 
x = 2; y = 3; z = 4;
matrix[(x, y, z)]
 
# Output: 88

Set Types: `set` & `frozenset`

A set in Python is an unordered collection of unique and immutable objects that support mathematical set theory operations. Since unordered sets do not map values to keys, sets are neither sequence nor mapping types [@lutzLearningPython2003]. Sets can only contain hashable (immutable) objects; mutable objects such as lists or dictionaries cannot be embedded in sets. To store compound values, use Tuples.

s = {1, 2, 3}
 
s.add([1, 2, 3])
# TypeError: unhashable type: 'list'
 
s.add((4, 5, 6))
# {1, 2, 3, (4, 5, 6)}

Sets themselves are mutable, too, and cannot be nested in other sets directly. To store sets inside other sets, you can use frozenset built-in to create an immutable set.

Similar to the List type, Set supports comprehension expression. The only difference is that Set uses curly brackets (Set literal {}) instead of square brackets (List literal []).

{ x ** 2 for x in [1, 2, 3, 4] }
# {1, 4, 9, 16}
 
{ x for x in 'Python' }
# {'P', 'h', 'n', 'o', 't', 'y'}
 
{ x for x in 'Apple' }
# {'p', 'A', 'l', 'e'}
 
{ x * 3 for x in 'Pythony' }
# {'PPP', 'hhh', 'nnn', 'ooo', 'ttt', 'yyy'}
# Notice that Set only can contain unique objects

Similar to dictionaries, sets are implemented internally with highly optimised hash tables.

File

Unlike many other programming languages, Python treats the file type as a built-in core type, which one can obtain a file instance by calling the open() method. In Python 3, the read and write operations treat files as Unicode encoded by default - unless you specifically pass in the b (for binary) flag.

Python has many built-in utilities for handling files. For example, the pickle module serialises objects, the struct tool packs and unpacks binary data, the JSON module converts Python objects to and from JSON syntax, and the shelve module provides keyed storage and access to Python objects.

Comparison and Equality Tests for Built-in Types

In Python, the equality test (\=\=) performs value equivalence - recursively for nested objects. The operator is conducts a reference test. Because Python caches immutable objects internally, two distinct short strings a = 'Python', b='Python' may pass the is test a is b. However, users should not rely on this compiler implementation feature and always perform the desired equality tests within the correct semantic contexts.

Different types have different definitions of equality and relative magnitude comparison results. For example, sets are equal if they contain the same items, dictionaries are equal if their sorted lists are equal, and lists and tuples are compared by each component from left to right.

Unlike JavaScript, Python does not support intrinsic conversion of types (apart from numbers - they will be converted to the highest precision) for comparison. Comparing variables of different types in Python will throw an error.

Python Virtual Environment

An environment in Python is an isolated context in which a Python program can be run. It consists of the Python interpreter and other required packages for the program. Using an environment is similar to the idea of Docker, which allows you to run an application in an isolated environment without polluting the host machines. For Python, this is especially important for operating systems that use multiple package managers for Python utilities, where the package managers may install conflicting packages and interfere with each other.

The venv module can be used to create a lightweight virtual environment on top of an existing Python installation.

python -m venv /path/to/new/virtual/environment

A virtual environment can be activated by running:

source _<venv>_/bin/activate

It prepends the venv to your PATH, so running regular Python commands, including pip and python, will invoke the virtual environment’s interpreter without being explicitly told so.

The Python Interpreter

Python interpreter comes with a collection of __builtins__ - a set of built-in functions, exceptions and other objects. Use help(__builtins__) to learn more about it.

Statement vs. Expressions

In Python, an expression is a programming construct that can be reduced to a value or a collection of symbols that jointly express a quantity. For example: 3 + 5 or [a.x for a in some_iterable]. A simple way to classify it is whether you can feed it to eval() - a valid expression can always be evaluated by evel(). Expressions can only contain identifiers, literals, and operators - including the call operator (), subscription operator [], arithmetic, and boolean operators.

Statements are everything else that can make up a line. They perform actions, that is, they do something. Statements are the smallest standalone element in an imperative programming language. The distinction between an expression and a statement is an important one in avoiding common coding mistakes:

#
# This is a common mistake in Python
 
L = [1, 2, 3]
L = L.append(4)
# `apend()` changes the list in place but itself returns None
# `=` makes a statement, not an expression
 
print(L)
# By assigning L to L.append(), you actually have lost the
# reference for the reasons above
 
#
# The correct way to do this is:
 
L = [1, 2, 3]
L.append(4)
print(L)        # Works as expected

Every valid expression can be used as a statement (called an expression statement).

Variable Naming Conventions

In Python, variable names that begin with a single underscore are not imported by a from module import * statement.

Useful Tools

PyDoc provides multiple ways of displaying documentation for built-in and imported modules and application scripts. It can start an HTTP server locally and provide a nice web UI with search functionality and auto-generated links, allowing you to click your way through the relevant modules in your application.

python -m pydoc -b

Functions

Unlike in compiled languages such as C++, Python def is a regular statement that assigns a function object to a name at runtime. Functions are first-class objects in Python (first-class object model) and can be passed around and stored in lists or dictionaries like any other Python object. Python functions are only evaluated at runtime when reached and are not compiled before the application starts. Furthermore, Python functions do not have to be fully defined before the application starts, as the code inside def is only evaluated when the functions are called later. The below syntax is valid in Python (not in traditional compiled programming languages):

# Define two different versions of the same function based on
# some conditions
 
if test:
  def func():
    ...
else
  def func():
    ...
...
func()
 
# You can pass functions as params to other functions
 
def indirect(func, arg):
  func(arg)

Similar to JavaScript, you can attach arbitrary information to functions in Python for later use:

def func()
  ...
func()
func.attr = value. # Attach attribute

OOP in Python

Class is the main OOP tool in Python. However, similar to the def statement, the class keyword in Python is an executable statement that assigns a special object to a name, not a declaration like in traditional OOP languages such as C++. This concept is similar to the class model in JavaScript - classes are factories that use constructors to create new instances.

Another distinctive characteristic is that classes are objects created when the application runs, typically when it is imported and read by the compiler. Like ordinary objects, classes in Python can be modified in place at runtime, and their instances will reflect any changes to the class definition. Like functions, class objects can have data attributes attached to them and shared by all instances of the classes. E.g., a counter, a flag, or other state shared by all instances.

Python classes are mostly namespace objects. The way their attributes are created is similar to Python modules and functions. When a class is imported, Python executes all the statements nested inside a class body, from top to bottom. Assignments that happen during this process create names in the class’s local scope and assign values to them, creating the namespace abject.

The namespace concept in Python is an important one. The inheritance search goes up the chain (references) but assignments only affect the instances themselves, leading to seemingly strange behaviour. For example:

class SharedDatte:
  x = 42
 
a = SharedDatte()
b = SharedDatte()
 
print(a.x, b.x)
 
# We can change the value of x for all instances in place at runtime
SharedDatte.x = 43
 
print(a.x, b.x, SharedDatte.x)   # Output: 43, 43, 43
 
# Assignment only affects the instance itself, not travelling up the 
# inheritance chain
a.x = 99
 
print(a.x, b.x, SharedDatte.x)   # Output: 99, 43, 43

Polymorphism in Python

Unlike statically typed languages, Python has a different philosophy of Polymorphism. Let’s look at an example:

def times(x, y):
  return x * y
 
times(3, 4)
# Output: 12
 
times('Python ❤️ ', 4)
# Output: 'Python ❤️ Python ❤️ Python ❤️ Python ❤️ '

This behaviour may surprise developers from a traditional statically typed programming background. However, it is intentional and a feature in Python. When programming in Python, one should not be concerned about the data types on which a specific piece of code operates. As long as the data types confirm the appropriate protocols the code expects, they should be allowed to benefit from the existing utilities (hence the increased expandability of the language). This is commonly known as duck-typing. The very nature of Python not declaring the types of variables implies that one should not rely on the types of objects (no type check) to function correctly (apart from special requirements).

References

Lutz, M. & Ascher, D. (2003) Learning Python. ‘O’Reilly Media, Inc.’
Wikipedia Contributors (2019). Python (programming language). [online] Wikipedia. Available at: https://en.wikipedia.org/wiki/Python_(programming_language).
Ramalho, L. (2022). Fluent Python, 2nd Edition. Beijing ; Boston Und Drei Andere: O’reilly

LLM Explainability & Interpretability

Wed, 04 Jun 2025 00:00:00 GMT

LLMs are known for, often criticised, and feared for their lack of explainability in their outputs and poor interpretability. Even when we peek into those black boxes and observe the steps taken to generate the final responses, we don’t understand how they ‘think’. This flaw had been a major showstopper for many real-world applications. For example, suppose a bank uses LLMs to decide whether a loan should be approved, but cannot explain the exact reasons behind the decision to the customer or the regulators. It will be very awkward.

Fast-forward to June 2025, three years after LLMs made the headlines, leading LLM providers started to expose LLMs’ internal reasoning process, hoping to ease the concern about using LLMs in critical decision-making processes. Google announced Thought Summaries, and Anthropic introduced an open-source tool to trace the thoughts of LLMs with attribution graphs in June 2025.

Referecnes

Anthropic.com. (2025). Open-sourcing circuit-tracing tools. [online] Available at: https://www.anthropic.com/research/open-source-circuit-tracing [Accessed 5 Jun. 2025].
Google AI for Developers. (2025). Gemini thinking. [online] Available at: https://ai.google.dev/gemini-api/docs/thinking#summaries [Accessed 5 Jun. 2025].

Accountability & Motivation

Mon, 10 Mar 2025 00:00:00 GMT

Extrinsic motivation has limits. Research shows that salary increases yield diminishing returns once employees earn enough to meet their needs. What truly drives people is intrinsic motivation -mastery or self-realisation - which varies by individual.

The book « The Psychology of Money » talks about the ultimate intrinsic value of money - being in control of one’s time and having the freedom to pursue one’s dreams. Being in control of one’s time is being in control of one’s life. But Do we all have to work hard, save, and quit our jobs? Not necessarily. To motivate employees, we can empower people to own their time and give them the freedom to do their best. But how? Ownership and accountability come to mind.

Accountability is a dirty word in business. ‘Holding someone accountable’ means we will blame you if something goes awry. We can all agree that it is not very motivating. In the book « The 12 Week Year », the authors give accountability another meaning: ownership, with detailed instructions on how to do that. How we translate business objectives to ownership of employees’ time and freedom to achieve something meaningful that aligns with employees’ vision is what leaders should strive to achieve.

References

Moran, B. and Lennington, M. (2013). The 12 Week Year. New Jersey: John Wiley & Sons.

Reasoning Models

Fri, 14 Feb 2025 00:00:00 GMT

Reasoning Models

Reasoning models were the stars of the show in the Gen AI world in the second half of 2024. OpenAI GPT-o1, o1 mini and o3, Google Gemini 2, Anthropic Claude 3.7, and DeepSeek R2, to name a few.

Rather than making larger and larger base models, like the trajectory from GPT-1 to GPT-3.5, AI companies are increasingly focusing on better reasoning capability through reinforcement learning on Chain-of-Thought (CoT) datasets. For anyone who is not new to Gen AI, CoT is not a new concept. It has been widely used in AI application development as a Prompt Engineering technique since the inception of LLMs.

This is a reasonable segue to better AI applications. Adoption has been hard; not everyone is an expert in prompting and fine-tuning LLMs. When equipped with better reasoning capabilities, LLMs are more useful out-of-the-box for the general public and downstream wrapper applications.

However, packaging too many ‘capabilities’ upstream has its consequences. CoT reasoning at the service level and integrated tool use (e.g., web search) make the model appear slower, as completing a list of sequential tasks inevitably takes longer. It also uses more tokens (OpenAI hides the reasoning steps from users but changes the token usage for reasoning steps), resulting in a more expensive, slower service for everyone.

Although DeepSeek R1 demonstrated that CoT can be part of training datasets, I believe (from the information released) that many other closed-reasoning models use extra processing layers. The extra reasoning power is not built into the text-predicting mechanism of vanilla LLMs.

Some argue that reasoning is the last step of human intelligence, and this may be the start of LLMs building so-called world views. It is a philosophical argument, and I do not dispute it.

However, I could not help but feel that this new direction is partly because AI companies are under pressure to (understandably) continuously release ‘better’ models. They are drifting away from the difficult task of creating fundamentally advanced foundation models. Sam Atman is right that AI adoption might not be transformative until AGI is achieved.

Besides, apart from complex tasks such as scientific research, factor checking and maths, not many tasks require tremendous reasoning capability out-of-the-box, and none can justify the eye-watering costs. What they need is a way to monetise their proprietary data. Suppose a firm does not have a robust data pipeline and data strategy. In that case, they are probably not ready to harness the power of reasoning models and to create value for themselves or their customers.

Even when high reasoning ability is crucial to a use case, other techniques, such as prompt engineering or an agentic workflow, can achieve the same with more tailored control and better performance at far lower costs. Besides, with the inherent shortcomings of the current LLMs’ inability to understand the real world with facts, reliable reasoning is very challenging. Instead of adding complex layers on top of existing LLMs, creating new architectures that can give future AI models world views might be far better for the future of AI.

Gen AI: The Future Is Agentic

Sat, 11 Jan 2025 00:00:00 GMT

Artificial Intelligence: A Modern Approach (Prentice Hall, 1995) is a modern classic that defines AI research as ‘_the study and design of rational agents.’

Agents are undoubtedly the most celebrated use case of Gen AI in 2024, with the emergence of a slew of frameworks and tools, such as Microsoft Magentic One, LangGraph, Vellum, Rivet, CrewAI, Amazon Bedrock Agents, OpenAI Swarm and OpenAI Agents Tools.

From Microsoft’s headline ‘The future is agentic’ with the announcement of The Magentic One to Google branding its flagship Gemini 2 as ‘Enabling the agentic era’, big techs are increasingly emphasising the creation of new agentic tools and equipping their frontier models with agent-friendly features.

Hallmarks of Practical Agentic Systems

Sat, 11 Jan 2025 00:00:00 GMT

It is widely touted that “the future of AI is agentic.” However, there is an intricate trade-off in going fully agentic: the more autonomous the agents in an application, the less reliable the application will be.

Going agentic is only powerful when the workflow is complex. Consider other techniques, such as augmenting LLMs with RAG systems and tools for simple use cases, and only use agents for the scenarios involving prompt chaining, routing, parallelisation, worker orchestration or evaluation optimisation.

To address the issue, AI companies either train more capable models to support agentic workflows natively or create specialised application frameworks such as LangGraph and Magentic One to help downstream AI application developers. Of course, you can use both.

2024 saw a few key features emerge, often considered the backbones and benchmarks of a practical agentic solution.

State & memory management & persistence

Memory can be used to remember a single conversation (a thread) or for information across multiple conversations. For example, information about the user, their preferences and past interactions with the LLMs. All this information can make future conversations between the user and AI more natural. It is more close to natural conversions between actual humans. It will feel strange if someone forgets the previous conversations whenever it’s your turn to speak. In this case, the dialogue between the two participants will not be able to carry on.

Implementation details differ with different frameworks, but the idea is the same. For example, in LangGraph, you can use Checkpointer to remember a single thread and the Store interface for long-term memory. In LangChain, you can use RunnableWithMessageHistory to facilitate chats between humans and LLMs.

Practitioners have tried to map the memory types of human brains to that of AI agents: semantic, episodic, and procedural. For AI agents, semantic memory often refers to information about a specific user. Episodic memory refers to the agents’ past actions. Besides using a permanent store, few-shot prompting is another convenient way to provide LLMs with episodic memories. Procedural memory refers to the agent’s system prompts or out-of-box model capabilities. Memories can be updated on the hot path or in the background, with a trade-off between performance and simplicity.

One caveat is that memory does not follow the ‘the more, the better’ rule. With less advanced models, the longer the context window filled with memory, the more likely the LLMs are to hallucinate or be forgetful. Conciseness is the key here. One commonly used technique is keeping track of a conversation summary instead of the entire conversation verbatim. After each new turn, you can use LLM to update the summary with any new info from the recent chat. This is closer to a real-world conversation, where you may not remember every word the other person said, but you remember the critical points and even resume the conversation when you meet the person next time.

Human-in-the-loop

The ability to intervene is essential for many real-world AI use cases, especially when AI safety, accountability, and regulatory requirements are involved. All popular agentic systems allow users to provide feedback, approve/refuse steps, and update the application state to influence AI agents’ behaviour.

Controllability

Fine-grained control over agents’ behaviour is key to developing high-performing agentic systems. Different tools have different opinions on this topic. Some empower engineers to design their agentic workflow from scratch, while some prescribe the orchestration part of the system.

Other considerations

LLMs are often better prompt authors than humans.

Using LLMs to construct or rewrite prompts for complex tasks proves hugely effective. In addition, combined with long-term memory, we can ask LLMs to continuously improve the system or application prompts (self-improvement) based on human-in-the-loop feedback.

Not all problems require an agentic solution. A simple LLM application with well-designed and optimised prompts can achieve a lot. Although powerful, agentic systems have the drawbacks of increased costs and degraded speed. It may be overkill in many use cases.

References

Anthropic (2024) The Anthropic Economic Index. 19 December 2024. https://www.anthropic.com/news/the-anthropic-economic-index?utm_source=tldrnewsletter [Accessed: 12 February 2025].

Google Agentspace

Fri, 27 Dec 2024 00:00:00 GMT

Google Agentspace

As a natural evolution of Google Enterprise Search and to harness the popularity of NotebookLM, Google announced Agentspace, repackaging the most practical use cases that emerged across businesses into one service. The main features of Google Agentspace are:

Multimodal RAG (NotebookLLM)

Like many other enterprise RAG solutions, users can securely upload documents and query about the uploaded info, whether asking direct questions, analysing data, reading charts and graphs, or summarising a slide deck.

What sets Google Agentspace apart is that it can use NotebookLM to generate podcasts (with a call-in feature as a paid offering), making the consumption of insights more engaging. Google also promises Google Search-like query results.

Pre-built connectors

It has many pre-built connectors for third-party vendors, such as JIRA, Slack, Microsoft Outlook, and Dropbox. This allows it to access enterprise data from other systems, often a significant take for AI adoption in many companies.

Expert agents

Google allows custom agents to automate bespoke business workflows. This powerful feature elevates Google Agentspace to a different level. It gives LLMs control over other systems, enabling them to generate insights, make recommendations, and take direct actions (with human-in-the-loop if necessary).

References:

Google.com. (2024). Google Agentspace. [online] Available at: https://cloud.google.com/products/agentspace.

Model Context Protocol (MCP)

Sun, 15 Dec 2024 00:00:00 GMT

Model Context Protocol (MCP) is an open-source protocol created by Anthropic to provide a standard way of connecting AI clients with data sources that often sit behind corporate firewalls or private data storage solutions.

Instead of building and maintaining different RAG connectors for purpose-built AI applications, an open standard on how applications provide context to LLMs greatly simplifies the process, improves AI response quality and system reliability, and reduces development costs.

As of December 2024, Anthropic’s Claude Desktop App can support local MCP servers. In addition to integrations built by Anthropic and contributed by the open-source community, many companies and tools have adopted MCP to make their systems available to AI clients.

References

Anthropic.com. (2024). Introducing the Model Context Protocol. [online] Available at: https://www.anthropic.com/news/model-context-protocol.

Magentic One

Tue, 12 Nov 2024 00:00:00 GMT

Quote

The future of AI is agentic.

– Microsoft

Magentic-one [@fourneyMagenticOneGeneralistMultiAgent2024] is a generalist agentic system built on top of AutoGen, Microsoft’s popular open-source agentic system.

Quote

It’s the difference between generative AI recommending dinner options to agentic assistants that can autonomously place your order and arrange delivery. It’s the shift from summarising research papers to actively searching for and organising relevant studies in a comprehensive literature review.

– Microsoft

Magentic-one consists of four agents: Orchestrator, WebSuffer, FileSuffer, Coder and ComputerTerminal. Users can use AutoGenBench, Microsoft’s agentic system evaluation framework, to mitigate risks from malicious Gen AI use cases.

References

Fourney, A., Bansal, G., Mozannar, H., Dibia, V. & Amershi, S. (2024) Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks. Microsoft Research. https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/.

Manual - Obsidian

Mon, 11 Nov 2024 00:00:00 GMT

Theme

Things Theme (w/ accent --pink: #ff82b2;)

Plugins

Zotero Integration

Download PDF Utility
Citation Formats
- Click Add Citation Format
  - Name: Insert Reference
  - Output Format: Formmated Biblibiography
  - Citation Style: Imperial College London - Harvard
- Click Add Citation Format
  - Name: Citation (Pandoc)
  - Output Format: Pandoc
  - Include Brakets: Yes
- Click Add Citation Format
  - Name: Citation (Harvard)
  - Output Format: Formmated Citation
  - Citation Style: Imperial College London - Harvard

Pandoc Reference List

Specify the ‘Path to biblibiography file’ to point to the bib file generated by Zotero
Citation Style: Imperial College London - Harvard
Cmd + P → Pandoc Reference List: Show reference list

Tasks

Examples (with the Things theme Extras)

[/] In Progress
[-] Cancelled
Done
[>] Forwarded
[<] Scheduled
[?] Question
[!] Important
[*] Star
[”] Quote
[l] Location
[b] Bookmark
[i] Information
[S] Savings
[I] Idea
[p] Pros
[c] Cons
[f] Fire
[k] Key
[w] Win
[u] Up
[d] Down

- [/] In Progress
- [-] Cancelled
- [x] Done
- [>] Forwarded
- [<] Scheduled
- [?] Question
- [!] Important
- [*] Star
- ["] Quote
- [l] Location
- [b] Bookmark
- [i] Information
- [S] Savings
- [I] Idea
- [p] Pros
- [c] Cons
- [f] Fire
- [k] Key
- [w] Win
- [u] Up
- [d] Down

Prompt Caching and LLM Memory

Thu, 29 Aug 2024 00:00:00 GMT

Claude allows [@anthropicPromptCachingClaude2024] users to cache frequently used contexts, such as conversations, coding snippets, questions about large documents or detailed instructions, between API calls to reduce costs by up to 90% and latency by up to 85%.

OpenAI provides a similar feature called Memory, which works similarly to Claude’s prompt cache under the hood. Instead of explicitly caching user prompts, it searches past conversations, documents, and interactions between users and ChatGTP. It extracts relevant information, feeding into GPT models as part of the prompt context.

Both mechanisms are similar to the LangChain concept of memory, albeit more integrated and slightly more sophisticated.

References

Anthropic (2024) Prompt caching with Claude. https://www.anthropic.com/news/prompt-caching.

Nemotron-4 340B

Sat, 20 Jul 2024 00:00:00 GMT

NVIDIA released the Nemotron-4 340B models under the NVIDIA Open Model License, a permissive licence that allows commercial use without attribution requirements.

The most distinctive feature of these models is that they are trained mainly on high-quality synthetic data. NVIDIA believes that this will lead to further advancement in AI development as it will eliminate many data-related issues such as rights, quality and preparation efforts.

NVIDIA plans to publish the datasets and to open source the synthetic data-generation pipeline it used to train the Nemotron-4 models, hoping to provide the AI community with more high-quality tools.

Compared to other open-source models such as Llama 3 70b, Mistral 8x22 and Qwen 2 72b Base, the Nemotro-4 family topped many performance benchmarks.

References

NVIDIA Technical Blog. (2024). Leverage the Latest Open Models for Synthetic Data Generation with NVIDIA Nemotron-4 340B. [online] Available at: https://developer.nvidia.com/blog/leverage-our-latest-open-models-for-synthetic-data-generation-with-nvidia-nemotron-4-340b/ [Accessed 20 Jul. 2024].

GitHub Copilot Extensions

Sun, 02 Jun 2024 00:00:00 GMT

GitHub Copilot Extensions

GitHub Copilot extensions allow partners and vendors to expand Copilot’s capabilities with vendor-specific tools and software knowledge and support. Developers can build, deploy, query documentation, examine resource status, access best practice code examples of vendor software and frameworks, and ask Copilot to take actions in external tools on users’ behalf, all without leaving the IDE, drastically improving the development flow. For example, with the GitHub Copilot Docker extension, developers can ask the Copilot to explain what containerisation is, how to create a docker file, to open a PR with the docker assets and to scan the code for vulnerabilities with Docker Scout.

References

Rodriguez, M. (2024). Introducing GitHub Copilot Extensions: Unlocking unlimited possibilities with our ecosystem of partners. [online] The GitHub Blog. Available at: https://github.blog/2024-05-21-introducing-github-copilot-extensions/ [Accessed 2 Jul. 2024].

Python Comprehension Expressions

Fri, 24 May 2024 00:00:00 GMT

Comprehension is a powerful feature that simplifies working with Python collections, such as lists, tuples and dictionaries (any object that supports iterable protocol).

List Comprehension (`listcomp`)

We can use list comprehension whenever we need to build a list from any iterable type. Although the same can be achieved using traditional for and while loops, comprehension expressions are more performant (with CPython’s C core), expressive (handling nested for loops with if conditions), and explicit (the sole intent of list comprehension is to build a list). List comprehensions can achieve anything map and filter functions are capable of.

Be careful not to abuse list comprehension, for example, executing code for its side effects. List comprehension should only be used to create a list.

colours = ['black', 'white']
sizes = ['S', 'M', 'L']
tshirts = [(colour, size) for colour in colours for size in sizes]  
 
tshirts 
# [('black', 'S'), ('black', 'M'), ('black', 'L'), ('white', 'S'),  ('white', 'M'), ('white', 'L')]

The var(s) that hold value(s) for the for loop in a listcomp have a local scope. You can use a := (Walrus operator) to make the values accessible after the express returns. For example:

colours = ['black', 'white']
sizes = ['S', 'M', 'L']
tshirts = [(colour, size) for colour in colours for size in sizes]  
tshirts_walrus = [last := (colour, size) for colour in colours for size in sizes]  
 
colour
# NameError: name 'colour' is not defined
last
# ('white', 'L')

Generator Expression

Syntactically nearly identical to listcomp (enclosed in parentheses instead of brackets), Generator Expressions (genexps) are more memory-efficient when working with large data objects - they yield items one by one using the iterator protocol instead of building out the entire list in memory.

colours = ['black', 'white']
sizes = ['S', 'M', 'L']
 
for tshirt in ((c, s) for c in colours for s in sizes):
    print(tshirt)
 
# ('black', 'S') 
# ('black', 'M') 
# ('black', 'L')
# ('white', 'S')
# ('white', 'M')
# ('white', 'L')

Origin of Comprehension

Although comprehension was mainly inspired by functional programming languages such as Lisp and Haskell, few modern OOP languages have similar features and concepts. For example, C# supports LINQ - a powerful extension that simplifies the code when working with complex collection types.

Developing LLM Products

Tue, 14 May 2024 00:00:00 GMT

Pitfalls to avoid when developing LLM products

LLMs are NOT databases

LLMs are not databases. They do not produce precise factual answers to questions, instead, they are ‘extremely good at telling you what a good answer to a question like that would probably look like.’ [@evansBuildingAIProducts2024]

References

Evans, B. (2024) Building AI Products. 8 June 2024. https://www.ben-evans.com/benedictevans/2024/6/8/building-ai-products [Accessed: 10 June 2024].

Liwen's Notes

Python Programming Language

Mutable vs Immutable Objects in Python

Dynamic Typing in Python

Garbage Collection & References

Data Types

Number

Sequence

Slicing

Mapping

Built-in Data Types

Boolean

Numbers

String

String Interning

List

Tuple

Dictionary

Set Types: set & frozenset

File

Comparison and Equality Tests for Built-in Types

Python Virtual Environment

The Python Interpreter

Statement vs. Expressions

Variable Naming Conventions

Useful Tools

Functions

OOP in Python

Polymorphism in Python

References

LLM Explainability & Interpretability

Referecnes

Accountability & Motivation

References

Reasoning Models

Reasoning Models

Gen AI: The Future Is Agentic

Hallmarks of Practical Agentic Systems

State & memory management & persistence

Human-in-the-loop

Controllability

Other considerations

References

Google Agentspace

Google Agentspace

Multimodal RAG (NotebookLLM)

Pre-built connectors

Expert agents

References:

Model Context Protocol (MCP)

References

Magentic One

References

Manual - Obsidian

Theme

Plugins

Zotero Integration

Pandoc Reference List

Tasks

Examples (with the Things theme Extras)

Prompt Caching and LLM Memory

References

Nemotron-4 340B

References

GitHub Copilot Extensions

GitHub Copilot Extensions

References

Python Comprehension Expressions

List Comprehension (listcomp)

Generator Expression

Origin of Comprehension

Developing LLM Products

Pitfalls to avoid when developing LLM products

LLMs are NOT databases

References

Set Types: `set` & `frozenset`

List Comprehension (`listcomp`)