Book Review: "Thinking, Fast and Slow" by Daniel Kahneman

I started reading the book Thinking, Fast and Slow by Daniel Kahneman in early 2024. This was initially recommended to me by a friend, and I became even more motivated to read it upon hearing positive things about it from colleagues at my previous job, as many of the subtleties described in the book are extremely relevant to the appropriate design of interviews, focus groups, and surveys of human subjects in social science research. However, because it is a long book and the middle of 2024 was made busier for me by moving back to Maryland, traveling a lot, and starting a new job (some of which I have discussed in a previous post [LINK]), I could not finish reading this book until much more recently. Because of this large gap between reading the initial 60% and remaining 40% of this book, I admit that I have since forgotten many details from the initial 60% of this book. Moreover, I started making notes to myself in this post based on that initial 60% because I assumed that I would be able to finish reading the remaining 40% soon afterwards and I would therefore remember the book as a coherent whole, but because that didn't happen, many of the notes that I have made in this post that were supposed to form the skeleton of this post now no longer make as much sense to me. For these reasons, this post may seem a bit more stilted than other book review posts in this blog and will likely seem stronger/more coherent when discussing the latter 40% of the book.

The book is a lengthy exposition of novel ideas in psychology & behavioral economics that were empirically validated by the author, most often in conjunction with his longtime academic collaborator Amos Tversky. The concluding chapter does a good job of recapitulating the main ideas of the book. Most of the book explores various facets of individual & group-based human behavior based on the idea that there are effectively 2 modes through which individuals process information, which the author refers to as Systems 1 & 2. System 1 "thinks fast", making snap judgments based on limited information, heuristics, and a bit of laziness, and is the aspect of thinking that drives most day-to-day reactions & decisionmaking, while System 2 "thinks slow", making more deliberate judgments with more of an effort to gather all relevant information but must in turn be consciously engaged and ultimately disengages from mental fatigue (in favor of System 1) if engaged for too long. The book also considers how individuals' typical behaviors when faced with outcomes that are certain competing with outcomes that have known or unknown probabilities deviate from behaviors idealized by microeconomic theories of expected utility, notably that while the commonly observed behavior choosing a certain gain with a lower value than the expected value of an uncertain gain can be explained to some degree by expected utility theory, the commonly observed behavior of choosing a gamble on losing outcomes with an expected loss of larger magnitude than a different certain loss cannot be explained by expected utility theory; this partly explains the risks that people take in business and can be explained in turn by how people in their perceptions tend to overestimate probabilities that are close to but not exactly 0 and underestimate probabilities that are close to but not exactly 1. Finally, the book partly explains notions of hedonic adaptation (the idea that one's sense of well-being is generally similar in many different good or bad medium- or long-term circumstances by adapting to those circumstances) by distinguishing how people rate pleasure or pain when experiencing those things versus in hindsight and shows how people's conceptions of their identities & well-being in the past, present, and future are intimately tied to their actual memories and their abilities to form & retain memories. These aspects of self-conception as well as perceptions of probability can also be tied to Systems 1 versus 2, as many seemingly shortsighted decisions or perceptions can be explained by System 1 making snap judgments lazily & using heuristics based on incomplete information.

Especially as I read the latter 40% of the book, I came to appreciate how many of the ideas of this book had permeated into other things that I had read & heard from others and that I had internalized into my own worldview & view of myself. Professionally, I could see how so many aspects of framing could be important when designing surveys & focus groups. Personally, I could see how especially as I have aged, I have in many cases consciously chosen to not worry too much about certain details and instead make decisions based on lazier heuristics because I didn't feel that the results of spending more mental energy making a decision based on System 2 would be worth the effort. At the same time, I have become more consciously aware of how my memories of things in my own life can be affected by the passage of time and by more recent events in my own life, and I have become more consciously aware of the deep entanglement between my perceptions of my own memories and the narratives that shape my perceptions of my own life & of the world. I thus feel more proud of maintaining detailed personal diaries where I take note (using System 2 as much as possible when considering things outside of the current moment) of how I feel about various things in the moment as well as in hindsight and carefully consider how & why my thoughts & feelings about different events in or aspects of my life have evolved over time. Moreover, I have become more aware over time of when I might be vulnerable (through System 1) to the power of suggestion or to a subconscious desire to align with groupthink, though given that it is System 1, I am not necessarily aware of these things until later (thinking about these things through System 2). Finally, especially over the last several years, I have come to see many things at a very broad conceptual/philosophical level, whether the experiences in my own life, the evolution of different aspects of human society, or the expansion of human knowledge, in terms of perdurantism [LINK from Wikipedia]; although I am not philosophically sophisticated enough to be able to think through & defend all of its implications, it intuitively makes sense to me to think about personal identities, feelings, people, and other things that can be said to exist, in terms of their existence in spacetime and not just in space at specific instants of time. Because of my philosophical inclination in this way, I was particularly pleased to see the author discuss the idea of time-integrated pleasure or pain and of looking at changing identities or overall life courses in terms of spacetime.

Although this book is not technical at the level of an academic journal article, it is fairly technical compared to most nonfiction books aimed at the general public, so I would say that it is aimed at a well-educated reader. That said, I do think that it is written with reasonable clarity for non-academic audiences. Additionally, the book covers many topics, and it is recommended to bear in mind the headings of sections that comprise groups of chapters, because otherwise, it is easy to lose track of the narrative of the book, especially because the book is long enough that I suspect that it would be impossible for most readers (even those who read books, including more technical nonfiction books, relatively quickly) to finish this book in one sitting. I would say that the concluding chapter is a nice way to reinforce the main points of the book in the reader's mind and that the details of each chapter can be treated as a reference when needed as opposed to forming a perfectly coherent narrative in the progression of chapters in the book.

It is important to remember that some aspects of this book are out of date. In some cases, that is just because this book was published in 2011 and had been written over many years before that; for example, the author gives an example of estimating the likelihood of choosing a particular major in college, but that example uses base rates that seem to be quite out-of-date. In other cases, the book is out of date because it is based on academic experimental work in psychology & behavioral economics, and other studies may find contradictory (either null or opposite) results to those presented in this book. The Wikipedia article about this book [LINK] discussed how most of the results from most of the studies discussed in one chapter (as an example) have been found to be not replicable, with the author afterwards admitting to putting too much faith in those studies and therefore falling prey to the same biases as those discussed in that chapter & elsewhere in the book. As a slightly different example, later parts of the book discuss the ideas of nudge theory and its seeming successes in public policy, but the Wikipedia article about nudge theory [LINK] has pointed out that later studies & meta-analyses have found that after correcting for publication biases in favor of positive results & against null results, nudging does not yield statistically significant (non-null) effects on human behavior; in this case, one of the primary researchers (who is named in this book as a collaborator of the author & pioneer of nudge theory) has made some counterarguments that I don't find convincing.

With these caveats in mind, I would still recommend this book to anyone interested in these ideas and with the patience to carefully consider them, though this may partly reflect my own biases in how I view issues of identity & the world. Follow the jump to see my other assorted & disjointed thoughts about this book.

Third Laptop: Lenovo ThinkPad P14s

My previous laptop was an ASUS UX331UN, and I wrote a post when I got it 6 years ago reviewing it [LINK]. It served me well, so I figured for a while that when it came time to replace it, I would most likely get another ASUS laptop. Over most of this year, I noticed progressively more often when using that laptop that it would make an odd whirring sound, and that would be especially noticeable when it would be shutting down. This motivated me to start considering to get a new laptop. More recently, the recent US presidential election result forced the issue for me, as I was concerned that the president-elect, once inaugurated, would impose heavy tariffs on many different countries that would significantly impede global supply chains that are essential for the manufacturing of electronic goods, including laptops. Thus, I thought carefully about and made the decision to buy a new laptop. Follow the jump to see more.

Some Dangers of Confusing "Changing One's Mind" with "Bayesian Updating"

Recent conversations with friends & colleagues about probability theory reminded me of conversations with a friend of mine in graduate school about the supposed virtues of making one's own reasoning in one's daily life more systematic through Bayesian inference. The basic idea, in rough qualitative terms, is that one's belief in a hypothesis can be quantified through a prior probability, and when one observes some data related to that hypothesis, one can use the probabilities of observing that data when that hypothesis does or does not hold to update one's belief in (becoming the posterior probability of) that hypothesis based on the data. An example of quantitative & qualitative explanations can be found on the site LessWrong [LINK]. However, even in graduate school and again more recently, I realized that it is very easy for one to talk oneself into believing that one is using systematic Bayesian reasoning while actually just rationalizing one's own prior beliefs & changes in beliefs after the fact. This can be illustrated mathematically in a few ways that are not exhaustive. Follow the jump to see more.

Disability, History, Wilderness, Natural Parks, and Urban Spaces (Part 2)

This post is a follow-up to a post [LINK] about an essay by the environmental history professor William Cronon, which in turn was about the ultimately delusional, dishonest, or hypocritical (the particular adjective depending on one's viewpoint) way that many Americans & Europeans since the 19th century have viewed the ideas of "wilderness" and of being close to said "wilderness". That essay, recommended to me by a friend, strongly resonated with me because of my own ambivalence, developed over the course of the 3 years that I physically lived in California with my opinions strongly shaped by my lifelong disability and events that happened to me related to that (particularly being hit by a car [LINK]), about the ways that people in the western half of the contiguous US value "wilderness" or natural parks that don't make a lot of sense to me or don't seem coherent to me based on how most natural parks in the US as designed today exclude people with disabilities in many different ways. Because I have written notes about these and other events & thoughts in my life consistently for the last several years, I think it makes the most sense to first structure this post chronologically to lay out the development of my ambivalent mindset toward the extent to which other people have a particular positive view of "wilderness" that they highly value and then summarize these points more coherently in another section. I should warn that the chronological narration is quite repetitive in writing only because similar ideas occurred to me in marginally different from different stimuli at many different points in my life. In any case, I think that presenting the chronological narration is the most honest way to present my mindset, because "showing my work" makes it much less likely to mislead anyone (including me, as my specific memories naturally become more hazy over time) into assuming that I have felt or thought a certain way for longer than I actually have. Follow the jump to see more.

Disability, History, Wilderness, Natural Parks, and Urban Spaces (Part 1)

Over the 3 years that I physically lived in California (as I worked remotely for UC Davis remotely from Maryland for 1 year before that), I became progressively more ambivalent about the reverential attitudes that many people in California and more broadly in the western half of the contiguous US (including the Northwest, Mountain West, and Southwest) have toward wilderness and natural parks and a little bitter that such reverence could directly be connected to the way that urban spaces in this part of the US feel far more neglected and bare-bones than urban spaces in the Northeast & Mid-Atlantic do; the bitterness is related to my personal need, as someone with a disability who does not drive, for spaces with good public transit, safely walkable paths, and dense mixed-use development. I had thought about these issues more especially during this year, but thus far, I had not considered writing blog posts about these thoughts. It was only after reading the essay "The Trouble with Wilderness" published in 1995 by William Cronon [LINK], recommended to me by a friend who thought that I might be sympathetic to the arguments in the essay, that I felt compelled to further flesh out & share my thoughts about these issues in this blog.

The essay explains the dichotomies & hypocrisies inherent in primarily middle-class and rich American perceptions of wilderness and how these attitudes arose. The author explains how until the 19th century in North America & Europe, and until even later in many other parts of the world, wilderness was seen by most urban cultures as dangerous, desolate, and leading people through such desolation to despair & amorality, the latter exemplified by Rudyard Kipling's popularization in the late 19th century of the savage "jungle law". Additionally, there were roots before the 19th century of the idea of spending time in wilderness as leading to religious experiences, but such experiences were clearly meant to evoke terrified awe (consistent with the little bit that I understand about Christianity regarding its emphasis on sin & guilt) as opposed to transcendent bliss, and they were associated with people exiling themselves from society for tough religious penance, whether in the cloisters of a monastery or in a forest far from the comforts of urban civilization. The author explains that the transformation of popular perceptions of wilderness from negative to positive came in the 19th century in Europe & North America, as the perceptions of religious experiences shifted to being more uniformly positive & comforting and simultaneously as local governments started building more amenities to tame wilderness into being a natural park for tourists; I suspect that this also coincided with Friedrich Nietzsche's work on the overman (the self-realized man striving for betterment in conjunction with enjoyment of the world) being interpreted as humanity having a greater degree of control over nature and with the rise of prosperity theology in the US, but I am not yet well enough read to comment intelligently on Nietzsche's work, and in any case, the author does not explicitly make such connections or arguments. The author describes how this shift in perception was reinforced at the same time specifically in the US by the promotion of frontier myths; I was previously familiar with how the frontier myth played into those reverential attitudes in the US toward wilderness, but I didn't make the connection until reading this essay of this myth to the way that the people, including Teddy Roosevelt, who pushed this myth & related reverence for natural parks were in fact rich white American men who grew up in urban comfort & benefited from industrialization and were rewriting the frontier myth in their own image, contradicting the reality that most people, including cowboys, who worked on the actual frontier were racially, sexually, or otherwise socioeconomically marginalized by the settled WASP-dominated society in the Northeast & Mid-Atlantic. The author ties this erasure of history to how the reverence for natural parks among many Americans who grow up in urban settings took root because of the combination of feeling alienated from industrial areas (which were genuinely dangerous & polluted places to live in the 19th & early 20th centuries) and being ignorant of what undeveloped land (wilderness) is really like. The author argues that such positive perceptions are counterproductive for understanding how humanity can actually live sustainably with nature as such attitudes unduly compartmentalize "nature" as being irretrievably separate from humanity, and when combined with negative attitudes about urban environments & rigid beliefs that humanity destroys everything that it touches in nature, this leads to the logical (within its own axioms) yet incredibly depressing nihilistic conclusion that humanity should cease to exist. The author emphasizes that comfortable & positive experiences in the wilderness are too often accessible only to rich people in urbanized areas (in the sense of having the time & transportation to get to natural peaks that have a lot of physical infrastructure & amenities built deliberately, irrespective of the level of luxury of a given private tour) and that this has been true since the 19th century in North America & Europe. Finally, the author argues that it would be better to appreciate & cultivate nature closer to home even in urban settings.

My friend was correct; I did indeed feel that this essay strongly resonated with me, as I have had similar thoughts about the effects of the frontier myth on popular reverence for natural parks in the US and about the myth versus reality of human infrastructure & amenities in natural parks being marketed as "true wilderness". It is worth noting too that the word "jungle" came from the Sanskrit/Hindi word "jaṅgala", which originally meant "desert" (emphasizing the aridity) and was later expanded to refer to any place hostile to human settlement, but its application to thick forests with overgrown understories was based on a misunderstanding by British colonizers in India in the 19th century (which does not surprise me); this is relevant to illustrating perceptions in other cultures of wilderness and of shifts in perception in the 19th century, as other scholars who have published similar essays or book chapters in books or collections edited by William Cronon have showed how the shift in English away from the word "jungle", which had negative connotations, to the word "rainforest" was associated with a softening of the popular image of such ecosystems (previously seen in Europe and by white North Americans as harsh & antithetical to humanity).

I have learned over the last few years of discoveries over the last few decades about the ways that indigenous societies in America (considered as a single continent), before European colonization, shaped environments that in the 20th century were assumed to be untouched wilderness. The shaping took the form of light-touch agroforestry, silviculture (forest cultivation), and polyculture (farming many plant or animal species together in ways that are sustainable due to those species' ecologically mutualistic relations, as opposed to the monoculture prevalent in industrialized farming). Examples include the Amazon rainforest [LINK] whose shaping supported large & diverse highly-developed cities [LINK], wildflowers in the deserts of California [LINK] making clear that even the term "wildflower" in that context is a misnomer that erases indigenous American work on cultivating those plants over many generations, and the temperate rainforests of British Columbia where biodiversity was much higher due to human selection near indigenous settlements than farther away [LINK]. These discoveries came after William Cronon published his essay, so I hope that he would be aware of these more recent discoveries (as he retired from his tenured faculty position only within the last few years). Learning about these things motivated me to learn a little more, from Wikipedia, about agroforestry, silviculture, and polyculture. They seem like promising ways to promote soil fertility, biodiversity, greater resilience against pests & natural disasters, ecological health, and human health. However, it is important to recognize the tradeoffs between these benefits and the need for extensive delicate & prolonged human labor, given the implications for our current population level & standard of living in the US, instead of uncritically romanticizing such practices as better in every way than current practices of monocultures & mechanized farms, especially if such romanticism is a reflexive opposition to white dominance in the various countries of America, with that opposition in turn arising from sympathy with indigenous peoples who have been & continue to be oppressed. In particular, livestock & machines currently struggle to work well with any farming method other than monocultures, so scalability would be a problem, though I am optimistic that AI tools could be paired with more carefully-designed machines to more effectively seed & harvest more complicated crop growths in forests or polycultures.

The essay by William Cronon didn't have as much about disability or the neglect of urban spaces, which are more salient to my experiences. Thus, the extent to which the essay resonated with me because of those issues was more because my mind was filling in those gaps. For this reason, I am making this post one in a multi-part series, with this part focusing more on the essay itself and more directly related issues of indigenous land cultivation (as William Cronon's treatment of indigenous issues, which is understandable given the state of popular knowledge & research in the US in 1995, is with a sad tone as if indigenous peoples in America had been completely wiped out & existed only in the past, ignoring the ways that indigenous peoples in America continue to preserve traditional land management practices & shape their lands accordingly, even if those things happen now on much smaller scales than they did before European colonization). The next post in this series will focus more on my experiences & thoughts from the standpoints of disability & urban neglect; I may have more posts afterwards only if there is a clear need to break the material into shorter posts and there is a clean way to separate the posts by topic.

My time at TRB CATE 2024

Last month, I attended the TRB 2024 Conference on Advancing Transportation Equity (TRB CATE 2024). The conference, which was held in Baltimore, was probably the most enjoyable conference that I have attended thus far (though I have not attended that many), for the following reasons mostly related to how good it was for networking. First, the number of attendees, which was around 500, was enough for almost all of the attendees to meet new people & network effectively; the presence of fewer attendees might lead to a situation where a large percentage of the attendees know each other, leading to other attendees feeling left out, while the presence of more attendees might lead to most attendees feeling overwhelmed by the prospect of meeting attendees unknown to them and therefore discouraged to do much other than spend time with known attendees (as often happens in the TRB Annual Meetings). Second, the conference was more focused on equity issues in transportation, so attendees were very passionate about this issue, and this led to more positive vibes & more fruitful discussions (in contrast to something like the TRB Annual Meeting, where the breadth of topics means that it is unlikely for someone working on pavement engineering to have much in common with someone working on public transit network redesigns). Third, conference organizers explicitly encouraged (at multiple points during the conference) attendees to network, and the schedule (especially including the presence of multiple poster sessions structured as receptions/mixers) included a lot of time for networking instead of being packed from start to finish with highly structured sessions.

I am especially grateful to my current employer for sponsoring my attendance given that I presented a poster about work from my previous job (and, interestingly enough, this was personally my first time presenting a poster in any professional setting). It reflects well on my current employer's willingness to take chances with their employees and supporting employees' presence at such conferences so that the employer's name is more publicized and the employee gets the professional development benefits of learning & networking. Additionally, I got to meet several of my colleagues in person, which felt even better given that most of us work remotely and many of them live in different parts of the US. As this conference happened during the second week of my current job, it really felt like an auspicious way to start this job.

Starting a Job at Cambridge Systematics

I am pleased to share that I have started a job at the transportation consulting company Cambridge Systematics as of 2 weeks ago, being based out of its office in the DC area (where I grew up). (Note that this company, which was founded in Cambridge, Massachusetts in the US, is a different company than the social media data mining company Cambridge Analytica, which was founded in Cambridge in the UK and was notable for its mining of user data from social media sites for the purpose of targeted political advertising in the 2016 presidential election in the US [LINK from Wikipedia].) The company takes on clients primarily from government agencies at the federal, state, and local levels to do work on long-range planning, travel demand forecasting, big data analytics & modeling for passenger & freight transportation, transportation safety planning, public transit planning & operations, and public transit & highway asset management. I will primarily be working on projects related to shared mobility, public transit, long-range planning, and transportation safety, but I will not necessarily be restricted to these areas.

I am excited to be working at this company, as it will give me a different perspective on transportation than I had within academia. Additionally, unlike in academia where I was focused on leading & managing a few projects over a long period of time in which those projects would lead to research products but might not necessarily have any more immediate tangible impact, in this company, I will be able to work on a wide variety of shorter-term projects that will have more immediate tangible impacts for people. I got into the transportation sector to help people with disabilities, and although many of the projects at this company might not directly relate to the needs of people with disabilities, I look forward to bringing up those issues where possible & applicable, using these projects to educate colleagues & clients about these issues where possible & applicable, and having tangible (even if incremental) positive influences on transportation for people with disabilities through the completion of these projects.

Reflection: Leaving UC Davis

This week is my last week as a postdoctoral researcher at the UC Davis Institute of Transportation Studies (ITS-Davis). I am glad that I was able to transition from physics to transportation policy within the setting of academia and to particularly to so at ITS-Davis, which is renowned for having multidisciplinary transportation research & education that has included an increasing focus on issues of equity & accessibility in transportation planning. I learned so much, not just about transportation per se (which, in a professional context, was totally new to me when I started this job) but also about hiring, advising, and managing graduate students, applying for grants, managing grant-funded projects, communicating with different audiences beyond academia in many different forms, working on projects that are not just academic research ending in a peer-reviewed journal article, forming & managing relationships with stakeholders from government agencies, community-based organizations, and other organizations, and expanding my professional network on my own. This ultimately became the right time for me to leave ITS-Davis, but I will be grateful for the experiences & opportunities that I had in it and for the people that I got to work with.

Finite Determinants of Linear Operators in Continuous Vector Spaces

Recently, I wondered whether it is possible for a linear operator in a continuous (infinite-dimensional) vector space to have a finite determinant. By "continuous vector space", I mean that the identity operator can be resolved for a complete orthonormal basis \( |\phi(x) \rangle \) for all \( x \) such that \( \langle \phi(x), \phi(x') \rangle = \delta(x - x') \) as \( \hat{1} = \int |\phi(x)\rangle\langle \phi(x)|~\mathrm{d}x \). If an operator \( \hat{A} \) has continuous matrix elements \( A(x, x') = \langle \phi(x), \hat{A}\phi(x') \rangle \), then it is easy to see that the conditions for its trace \( \operatorname{trace}(\hat{A}) = \int A(x, x)~\mathrm{d}x \) to be finite are that the integral must converge, so the "function" \( A(x, x) \) must asymptotically approach 0 strictly faster than \( 1/x \) as \( |x| \to \infty \) and must at most have singularities at finite points \( x_{0} \) that diverge strictly slower than \( 1/|x - x_{0}| \). This can be seen as the continuum limit of a sum over the diagonal. However, the determinant is harder to express in this way because it involves products over diagonals & subdiagonals that are harder to express in a continuum space.

For this post, I will only consider Hermitian positive-definite operators. The conditions that I will list for which the determinant exists for such operators are sufficient for the determinant to exist, but I am not convinced that they are necessary. If such operators have an eigenvalue decomposition \( \hat{A} = \int a(x) |\phi(x)\rangle\langle \phi(x)|~\mathrm{d}x \) where the vectors \( \{ |\phi(x) \rangle \} \) form a complete orthonormal basis and the eigenvalues satisfy \( a(x) > 0 \) for all \( x \), then one can make use of the identity \( \ln(\det(\hat{A})) = \operatorname{trace}(\ln(\hat{A})) \) to say that \( \ln(\det(\hat{A})) = \int \ln(a(x))~\mathrm{d}x \). For the right-hand side to converge, then \( \ln(a(x)) \) must asymptotically approach 0 with \( x \) as \( |x| \to \infty \) strictly faster than \( 1/x \), which means that \( a(x) \) must asymptotically 1 with \( x \) as \( |x| \to \infty \) strictly faster than \( \exp(1/x) \) (which is not the same as \( e^{-x} \)), and \( \ln(a(x)) \) can at most have singularities at finite points \( x_{0} \) that diverge strictly slower than \( 1/|x - x_{0}| \), which means that \( a(x) \) must either diverge to \( \infty \) strictly slower than \( \exp(1/|x - x_{0}|) \) or drop to 0 strictly slower than \( \exp(-1/|x - x_{0}|) \). For example, \( a(x) = \exp(1/(x^{2} + x_{0}^{2})) \) fits the bill; note that this is not the same as the Gaussian kernel \( \exp(-(x^{2} + x_{0}^{2})) \). Intuitively, this condition makes sense, because for a finite-dimensional diagonal matrix as the dimension becomes arbitrarily large, the diagonal elements must mostly be exactly or very close to 1 for the determinant to not grow arbitrarily large with the dimension.

In finite-dimensional vector spaces, it is also easy to compute the determinants of triangular matrices simply as the products of the diagonal elements. (This is why the determinant is most often computed by an algorithm like first computing the LU decomposition and then taking the product of the diagonal elements of the upper-triangular matrix, which for an \( N \times N \) matrix involves \( O(N^{3}) \) operations, as opposed to the Leibniz formula involving every permutation which involves \( O(N!N) \) operations.) In infinite-dimensional vector spaces, a matrix that is triangular in a countable basis can have the determinant computed similarly as in finite-dimensional vector spaces; if an operator \( \hat{A} \) in that basis has elements \( A_{ij} \), then using the definition \( \ln(|\det(\hat{A})|) = \prod_{i} \ln(|A_{ii}|) \), the determinant converges as long as the diagonal elements \( |A_{ii}| \) are mostly exactly or very close to 1, specifically such that as \( |i| \to \infty \), \( \ln(|A_{ii}|) \) decays to 0 strictly faster than \( 1/i \). (Note that \( i \) is an integer index written in slanted font, not the imaginary unit \( \operatorname{i} \) written in upright font.) However, I am not sure how to generalize this to operators that are expressed as triangular matrices in continuous bases.

Transitioning from microscopic to macroscopic and quantum to classical regimes

I recently read two things that were of interest to me having previously worked in physics. One was an article in The New Yorker magazine [LINK], in which the author does a good job of going over the successes of mathematical modeling in the physical sciences and contrasting this with the limitations of mathematical modeling in public health (showing, for example, how many models of the spread of contagions fail when governments & societies take fast & drastic collective actions to limit the spread), the failures of mathematical models in social sciences where the outputs of those models can create feedback loops with public sentiment (for example in political polling), and the way that many people who use machine learning models in different domains expect the fancy curve-fitting of those models to represent fundamental understanding when that might not really be so. The other was a journal article published in Physical Review Letters [LINK] about how it can be possible to test the extent to which a massive (as opposed to massless) object which exhibits the dynamics of a simple harmonic oscillator and prepared in a quantum coherent state can be tested for deviations from classical behavior using a protocol that does not depend on the mass of the object (although I question this given that the protocol depends on timed measurements that depend on the frequency of oscillation, and in many physics contexts the frequency does depend on the mass as \( \omega = \sqrt{k/m}\), but this is somewhat of a quibble). These two things got me to think about something that I realized I never got out of many years of formal undergraduate & graduate education in physics. This can be illustrated with the following example.

In introductory physics classes that focus on Newtonian mechanics, a prototypical problem involves a block, modeled as a point mass, sliding (with or without friction) down a fixed triangular incline in the constant gravitational field of the Earth. In the context of those classes, instructors will be careful to note that this is merely a model, and corrections could come from the inclusion of the variation of the Earth's gravitational field & surface curvature, the technical possibility of moving the triangular incline (which must be much more massive than the block in question), the shape of the block, variations in the touching surfaces, air resistance, et cetera. In later classes, instructors may point out corrections due to special relativity (i.e. the speed of light) and general relativity (as it relates to the Earth's gravitational field).

However, in later classes about quantum mechanics & statistical mechanics, instructors explain how different the models are from models of Newtonian mechanics at human scales, but they often promise that appropriate treatments of aggregates of microscopic constituents can consistently recover results from Newtonian mechanics, yet this promise is almost never fulfilled. In particular, wavefunctions that describe pure states of single microscopic particles are quite far removed from the simple dynamical variables describing blocks on inclined planes, although statistical mechanics can probabilistically describe the solid states of the block & inclined plane as well as the gaseous state of the surrounding air, it is not usually extended to describe the dynamics of the block sliding down the inclined plane. For example, if a block sliding down a fixed inclined plane of horizontal angle \( \theta \) in a uniform gravitational field is described as having equations of motion \( m\ddot{x} = mg\sin(\theta) \) where the \( x \)-axis is defined as pointing downward parallel to the slope of the inclined plane for increasing \( x \) and the \( y \)-axis points outward in the normal direction from the inclined plane, then I wish to see corrections of the form \( m\ddot{\vec{x}} = \sum_{\mu = 0}^{\infty} \sum_{\nu = 0}^{\infty} \hbar^{\mu} k_{\mathrm{B}}^{\nu} \vec{f}^{(\mu, \nu)} \) where the lowest-order term is \( \vec{f}^{(0, 0)} = mg\sin(\theta)\vec{e}_{x} \). I have never seen these sorts of quantum or statistical corrections to Newtonian equations of motion in simple (in the context of Newtonian mechanics) systems. Similarly, it is rare to see how quantum or statistical mechanical systems can, in appropriate limits, reproduce classical systems; I can only think of the quantum coherent state of the simple harmonic oscillator as well as how the Moyal bracket in the phase space formulation of quantum mechanics reduces to lowest order in \( \hbar \) to the Poisson bracket, and in the latter case, intuitive construction of the quantum phase space quasiprobability function is made more difficult (compared to construction of a classical phase space probability density function, as I did in a post [LINK] from a few years ago) by the fact that unlike the classical phase space probability density function, the quantum phase space quasiprobability function cannot be arbitrarily localized in phase space, it can take on negative values for certain wavefunctions, it is compressible in phase space with respect to its own evolution over time, and it is not obvious how it should look for a system of many particles constituting a macroscopic object like a block (in contrast to a classical phase space probability density function, which for such a system could just be a product of Dirac delta functions localizing each microscopic constituent to a point in phase space).

These considerations reminded me of a discussion I had last year with friends from college, who also did course 8 (physics) with me. We came to a consensus that while people who do not become physics majors should, as usual, get exposure to Newtonian physics and the basics of electricity & magnetism, people who become physics majors should have a curriculum over 3-4 years that exhibits a sensible conceptual progression. In particular, after seeing Newtonian mechanics, such students should then be exposed to Lagrangian & Hamiltonian formulations of classical mechanics. The Lagrangian formulation of classical mechanics should then be used to develop intuitions about mechanical waves, which in turn can lead to introductions to classical field theory and development of classical electromagnetic theory as a rich example of a classical field theory. (I would also personally recommend using the introduction of mechanical waves to introduce the linear algebraic treatment of waves and then reintroduce the linear algebraic treatment of waves into the treatment of linear classical field theories in general & linear classical electromagnetic theory in particular.) The Hamiltonian formulation of classical mechanics should then be used to develop intuitions about probability distributions in classical mechanics, which in turn can be used to develop intuitions about statistical mechanics. Optionally, at this point, the Hamiltonian formulation of classical mechanics can also be used to develop intuitions about nonlinear dynamics & chaos theory, but while this is good for the broader education of physics students, it is less immediately relevant for the introduction of quantum theory to come soon after (because quantum mechanics is linear). Finally, only after these things happen should quantum theory be introduced, such that there are clear connections of the wavefunction formulation of quantum mechanics to mechanical waves, the phase space formulation of quantum mechanics to classical phase space probability distributions, and the linear algebraic framework of quantum mechanics to linear algebraic treatments of classical field theories (including linear classical electromagnetic theory); this will ensure that students understand how ideas like superposition, interference, rotation through a Hilbert space, statistical uncertainty, and related ideas are not unique to quantum mechanics (which is unfortunately too often a consequence of the way quantum mechanics is typically introduced in undergraduate curricula, at least in the US). We also came to a consensus that in each course, there should be clear explanations of what prototypical systems are analytically solvable, what prototypical systems are not analytically solvable, and why (in each case).