from:	Zheng, Guohe <...>
to:	Dan Zheng <...>
date:	Nov 3, 2022, 4:44 PM
subject:	Acceptance Speech

Acceptance Speech of: 2022 IFLTA Distinguished World Language Advocate Award

It is a great honor to receive the 2022 IFLTA Distinguished World Language Advocate Award. The honor came to me as a surprise because I already retired in May from Ball State University, after teaching there for 25 years. I would like to thank Board members of AITJ for nominating a retiree for this honor.

Given the nature of the Advocate Award, today’s ceremony provides an occasion for me to look back at the paths that Japanese Education in Indiana has gone through in the past quarter century.

A few lines from my CV have highlighted the ups and downs of the paths. For example, one line says that “in the 3 years from 1999-2001, my colleague Dr. Tomizawa and I received $233,000.00 from Indiana Department of Education to train Indiana teachers in the development of Japanese culture curriculum in Indiana Schools including funds to take participating teachers and school administrators to Japan.” That project was completed in the summer right before 911. In retrospect, the pre-911 years were a golden age for World Language Education. Grant money seemed to be always available. All you need to do was to submit a proposal.

But 911 changed everything. And 911 was followed by the 2008-2009 recession, by the subsequent budget-cut everywhere. World languages, particularly less-taught languages such as Japanese, were on top of the budge-cut list, even program elimination. Moreover, due to the rise of China around the time, Japanese programs came to face the competition from emerging Chinese programs.

But Japanese programs in Indiana demonstrated great resilience in times of difficulties. This is shown by another line of my CV. In 2013, I co-edited a volume entitled Embracing the new phase in Japanese education in the United States, the Proceedings of the 23rd Annual Conference of the Central Association of Teachers of Japanese. That volume included stories of how some Japanese programs were phased out in Indiana, or forced to switch to Chinese, but it included more stories of how Japanese programs were retained with increased enrollment coming unexpectedly from students from China, or with petitions by students and parents, out of their love of their engaging and inspiring teachers of Japanese. Another way to cope with the challenge was the launching of dual credit Japanese in Indiana in 2013, which attracted many high school students. I am very proud to have played a role in creating and supervising Indiana Japanese dual credit and in doing so have had the good fortune to know many excellent high school teachers and learned so much from them, particularly their creative application of technology in the classroom.

Covid-19 in the past 3 years proved another major challenge to us all. I am proud to report that there was no interruption of the AITJ-sponsored annual high school student Japanese competition called JOI, Japanese Olympiad of Indiana. In fact, the online JOI generated more enthusiasm and creativity from the students.

As I stand here at the end of my career, I want to tell teachers mid-way in their career, but particularly teachers who have just started their career, that, I am confident that, even though you may have new challenges down the road, the future of Japanese education in Indiana is bright. My confidence comes from my personal journey of the past quarter century. Thank you all.

Swift-focused

• ML = machine learning
• PL = programming languages

The fields of ML and PL view the world from different perspectives.

<system 1 vs system 2>

ML is probabilistic and good at making suggestions, but has no guarantees built-in about properties of their predictions

PL techniques are rule-based (type systems, formal verification, etc) and guaranteed correct, but often do not scale to real world scenarios (production languages like C++, not enough of a sound solid base with nice properties to start with, need to build toy languages)

<link to Lucas’ talk at PLDI about GitHub Copilot>

It’s common for PL people to be skeptical of ML techniques, and vice versa.

However, one in which I learned on my current team is learning about how ML and PL can help each other.

<PL: good training data for ML, known oracles like type checkers and theorem provers>

<ML: allow producing lots of suggestions>

<Cite Prof Mayur Naik’s talk, slides about system 1 vs system 2>

View ML and PL not as in competition (actively bad) or orthogonal (neutral fields going in their own direction) but a positive feedback loop: ML helps PL be better at helping ML, etc

• ML supports constrained decoding: can limit the vocabulary for the next token to autoregressively regress.

• Maieutic prompting: a way to turn soft predictions from LLMs into more confident facts via clever self-consistency checking. Also enables getting confidence intervals on predictions, by recursively applying the approach to justify explanations and their negations. A good example of turning soft predictions into harder facts (not guaranteed, but quantified higher confidence).

• Consideration: models are good at learning correct syntax, robust against typos. Want to use PL for the biggest value-add, e.g. add data that provides the greatest value and is most difficult for model to learn by itself. (Consider adding an 2D plot of difficulty vs value).

After I joined this team, it took me a year to feel comfortable. … The nature of the work felt so different: much of ML (these days, at least my work) is transforming data and running command line invocations.

However, talking to expert researchers helped me feel better over time: learning about what makes them excited helped me become excited and productive.

And then I learned about the connections between my past and current work, how ML and PL can help each other. connecting the dots

Meta point: having multiple areas of expertise is not zero sum, total knowledge is more than sum of the parts, interesting insights and connections are waiting to be discovered – the best of both worlds

Good researchers can dive in and find interesting aspects of any subject!

• Japan is part of the Chinese cultural sphere
• They learned and borrowed kanji (Chinese characters)

China → Japan

• Japan was first modernized in the Meiji Restoration
• Many ideas and actual Chinese words were first invented by the Japanese (need sources)
  • Examples: society, science, chemistry, time(?), etd

What next?

More culture is coming from Japan into China in recent years

What about the other direction? I really do not know

$g$ constant

Trying to find one number

Programming languages

• Turing tarpit
• Escape from tarpit by comparing relative expressiveness
  • Non-trivial (local) transformation = A is more expressive than B
• What is the productivity number?
  • Hard to define, productivity is subjective and depends on complex human variables

Compression

• Kolmogorov complexity: reduce a program without losing information

Prediction

• Are some models better than others?
  • Character-model vs token-model vs tree-structured model

Humor

• Humor requires prediction, or can be formulated as seq2seq problem
• Humor requires compression: some jokes are best concise and bite-sized

Acknowledgements

Denys

• Requires three auxiliary types
  • Abstract box type
    • If protocol: it’s okay I guess
    • If class: need abstract class fatalError stubs
  • Concrete box type
    • Forwarding shims
  • User-facing AnyXXX type
    • Stores an abstract box, implemented as a concrete box

Possible to type-erase generic parameters?

• Seems hard if you try to store it, can’t do TensorN<Shape = _, Scalar = _>
• But wait… possible with closures! Introducing closure-based type erasure

Closure fusion

Demonstrates successful optimization across composition of type-erased calculations using functions..

https://gist.github.com/dabrahams/a44a2a078d8556ef9c2ff2c56e83e62a

• Note: closures are like callAsMethod function. This works best for types that have one main purpose: e.g. tensors are indexers, functions like + can be implemented in this paradigm.
• There is some relation to closures vs objects. Check out the Koan: https://wiki.c2.com/?ClosuresAndObjectsAreEquivalent

Type erasure survey

https://gist.github.com/dabrahams/c75760b7ed36dd4f039f3679ede825ea

• Turing tarpit
• Escape from tarpit by comparing relative expressiveness
  • Non-trivial (local) transformation = A is more expressive than B

What is the productivity number?

Acknowledgements

Denys

Start (small brain)

Lexing -> Parsing ->

Bigger brain

Why spend time on parsing?

What matters is:

• Definition of AST as an ADT
• All compiler functionality (interpret, compile, pretty-printing) are recursive functions with a big switch-case over the ADT

Conal Elliott

Commutative diagram

• Program
• Hardware
• Denotation (essence)

Back to the real world

What kind of compiler job should I look for?