+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+NumPy vs APL¶
-
+
- Going slow and easy, we have an hour +
- APL NumPy side-by-side +
- Goal: aware of APL as another tool +
- Non-goal: switch +
- Cover briefly then come back for Qs +
- Intentionally left out details so please ask as I go +
+
+
+
+
+
+
+
+
+About me¶
-
+
- Applied math in school +
- Knew APL from YouTube +
- Learned APL on the job last Jan (https://bcaresearch.com customer of Dyalog) +
+
+
+
+
+
+
+
+
+What we'll do¶
-
+
- Simple Moore's Law model +
- Real data (CSV) +
- Perform linear regression and predict exponential growth using ordinary least squares (tldr line of best fit) +
-
+
- Compare exponential growth constants between models +
- Save and share:
-
+
- NumPy zipped file
*.npz
+ - APL namespace file
*.apln
+ - CSV +
+ - NumPy zipped file
- Chat, happy little demos? +
Modified from https://numpy.org/numpy-tutorials/content/mooreslaw-tutorial.html
+
+
+
+
+
+
+
+
+
+What we need¶
-
+
- Python (packages):
-
+
- NumPy +
- Matplotlib +
+ - Dyalog APL:
-
+
- SharpPlot (included) (OOP) +
- Py'n'APL (optional, external, open source) +
+
+
+
+
+
+
+
+
+
+
+APL trigger warning
+-
+
- Butchered NumPy-style APL for comparison +
- APL users, sorry +
+
+
+
+
+
+
+In [1]:
+
+
+
+
+
+import matplotlib.pyplot as plt
+import numpy as np
+from pynapl import APL
+apl = APL.APL()
+run = apl.eval
+
+
+
+
+
+
+
+
+
+
+
+Dyalog APL Version 20.0.51290 +Serial number: UNREGISTERED - TechPreview - not for commercial use ++-----------------------------------------------------------------+ +| Dyalog is free for non-commercial use but is not free software. | +| A basic licence can be used for experiments and proof of | +| concept until the point in time that it is of value. | +| For further information visit | +| https://www.dyalog.com/prices-and-licences.htm | ++-----------------------------------------------------------------+ +Copyright (c) Dyalog Limited 1982-2025 ++
+
+
+
+
+
+ ⎕PW←32767
+ {}2⎕FIX'file:///home/user/.cache/pypoetry/virtualenvs/non-package-mode-VTfHNGXA-py3.12/lib/python3.12/site-packages/pynapl/IPC.dyalog'
+ {}2⎕FIX'file:///home/user/.cache/pypoetry/virtualenvs/non-package-mode-VTfHNGXA-py3.12/lib/python3.12/site-packages/pynapl/Py.dyalog'
+ infile←'/tmp/tmpvhlhx80a'
+ outfile←'/tmp/tmpt7x602na'
+ Py.StartAPLSlave infile outfile
+
+
+
+
+
+
+
+
+
+
+Model¶
Moores_law(year) = (e^B_M) × (e^(A_M × year))
A_M = (log 2) ÷ 2
B_M = (log 2250) - A_M × 1971
+
+
+
+
+
+
+In [2]:
+
+
+
+
+
+ # exp←*
+ # log←*○
+ A_M = np.log(2) / 2
+run("A_M ← (⍟ 2) ÷ 2")
+
+ B_M = np.log(2250) - A_M * 1971
+run("B_M ← (⍟ 2250) - A_M × 1971")
+
+ Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)
+run("Moores_law ← { year←⍵ ⋄ (* B_M) × *(A_M × year)} ⋄")
+
+[A_M, B_M], run("A_M B_M")
+
+
+
+
+
+
+
+
+Out[2]:
+
+
+([np.float64(0.34657359027997264), np.float64(-675.3778609466276)], + [0.3465735902799726, -675.3778609466276])+
+
+
+
+
+
+
+
+
+In 1971, there were 2250 transistors. Use Moores_law to predict how many in 1973.
+
+
+
+
+
+
+In [3]:
+
+
+
+
+
+ML_1971 = Moores_law(1971)
+ML_1973 = Moores_law(1973)
+print("In 1973, G. Moore expects {:.0f} transistors on Intels chips".format(ML_1973))
+print("This is x{:.2f} more transistors than 1971".format(ML_1973 / ML_1971))
+
+
+
+
+
+
+
+
+
+
+
+In 1973, G. Moore expects 4500 transistors on Intels chips +This is x2.00 more transistors than 1971 ++
+
+
+
+
+
+
+In [4]:
+
+
+
+
+
+run("""
+ML_1971←Moores_law 1971
+ML_1973←Moores_law 1973
+
+_←'.format←⍕'
+⎕←'In 1973, G. Moore expects ',(1↓0⍕ML_1973),' transistors on Intels chips'
+⎕←'This is x',(1↓2⍕ML_1973÷ML_1971),' more transistors than 1971'
+""");
+
+
+
+
+
+
+
+
+
+
+
+In 1973, G. Moore expects 4500 transistors on Intels chips +This is x2.00 more transistors than 1971 ++
+
+
+
+
+
+
+
+
+
+
+
+Loading historical data¶
-
+
- Transistor count each year in CSV +
- Inspect before load +
- Save year & count columns to
data.
+
Print first 10 rows of transistor_data.csv. The columns are
| Processor | +MOS transistor count | +Date of Introduction | +Designer | +MOSprocess | +Area | +
|---|---|---|---|---|---|
| Intel 4004 (4-bit 16-pin) | +2250 | +1971 | +Intel | +"10,000 nm" | +12 mm² | +
| ... | +... | +... | +... | +... | +... | +
+
+
+
+
+
+
+In [5]:
+
+
+
+
+
+! head transistor_data.csv
+
+
+
+
+
+
+
+
+
+
+
+Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area +Intel 4004 (4-bit 16-pin),2250,1971,Intel,"10,000 nm",12 mm² +Intel 8008 (8-bit 18-pin),3500,1972,Intel,"10,000 nm",14 mm² +NEC μCOM-4 (4-bit 42-pin),2500,1973,NEC,"7,500 nm",? +Intel 4040 (4-bit 16-pin),3000,1974,Intel,"10,000 nm",12 mm² +Motorola 6800 (8-bit 40-pin),4100,1974,Motorola,"6,000 nm",16 mm² +Intel 8080 (8-bit 40-pin),6000,1974,Intel,"6,000 nm",20 mm² +TMS 1000 (4-bit 28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm² +MOS Technology 6502 (8-bit 40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm² +Intersil IM6100 (12-bit 40-pin; clone of PDP-8),4000,1975,Intersil,, ++
+
+
+
+
+
+
+In [6]:
+
+
+
+
+
+run("""
+⎕←↑'UTF-8'∘⎕UCS¨⎕UCS¨⎕SH'head transistor_data.csv'
+⎕←''
+⎕←↑⎕SH'head transistor_data.csv'
+""");
+
+
+
+
+
+
+
+
+
+
+
+Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area +Intel 4004 (4-bit 16-pin),2250,1971,Intel,"10,000 nm",12 mm² +Intel 8008 (8-bit 18-pin),3500,1972,Intel,"10,000 nm",14 mm² +NEC μCOM-4 (4-bit 42-pin),2500,1973,NEC,"7,500 nm",? +Intel 4040 (4-bit 16-pin),3000,1974,Intel,"10,000 nm",12 mm² +Motorola 6800 (8-bit 40-pin),4100,1974,Motorola,"6,000 nm",16 mm² +Intel 8080 (8-bit 40-pin),6000,1974,Intel,"6,000 nm",20 mm² +TMS 1000 (4-bit 28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm² +MOS Technology 6502 (8-bit 40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm² +Intersil IM6100 (12-bit 40-pin; clone of PDP-8),4000,1975,Intersil,, + +Processor,MOS transistor count,Date of Introduction,Designer,MOSprocess,Area +Intel 4004 (4-bit 16-pin),2250,1971,Intel,"10,000 nm",12 mm² +Intel 8008 (8-bit 18-pin),3500,1972,Intel,"10,000 nm",14 mm² +NEC μCOM-4 (4-bit 42-pin),2500,1973,NEC,"7,500 nm",? +Intel 4040 (4-bit 16-pin),3000,1974,Intel,"10,000 nm",12 mm² +Motorola 6800 (8-bit 40-pin),4100,1974,Motorola,"6,000 nm",16 mm² +Intel 8080 (8-bit 40-pin),6000,1974,Intel,"6,000 nm",20 mm² +TMS 1000 (4-bit 28-pin),8000,1974,Texas Instruments,"8,000 nm",11 mm² +MOS Technology 6502 (8-bit 40-pin),4528,1975,MOS Technology,"8,000 nm",21 mm² +Intersil IM6100 (12-bit 40-pin; clone of PDP-8),4000,1975,Intersil,, ++
+
+
+
+
+
+
+
+
+Only need columns 2 & 3
+np.loadtxt:
-
+
delimiter = ','
+usecols = [1,2]: import cols 2 & 3
+skiprows = 1: skip header row
+
+
+
+
+
+
+
+In [7]:
+
+
+
+
+
+data = np.loadtxt("transistor_data.csv", delimiter=",", usecols=[1, 2], skiprows=1)
+
+
+
+
+
+
+
+In [8]:
+
+
+
+
+
+run("""
+csv←(⎕CSV⍠'Separator' ',')'transistor_data.csv'⍬ 4
+data←1↓csv[;2 3]
+data←1⊃⎕CSV'transistor_data.csv'⍬(0 4 4 0 0 0)1
+""");
+
+# csv = np.loadtxt("transistor_data.csv") # Value Error: could not convert string to float
+# data = csv[1:, 1:3]
+
+
+
+
+
+
+
+
+
+-
+
- Assign
datatoyearandtransistor_count
+ - Print first 10 values (
[:10])
+
+
+
+
+
+
+
+In [9]:
+
+
+
+
+
+year = data[:, 1] # grab the second column and assign
+transistor_count = data[:, 0] # grab the first column and assign
+
+print("year:\t\t", year[:10])
+print("trans. cnt:\t", transistor_count[:10])
+
+
+
+
+
+
+
+
+
+
+
+year: [1971. 1972. 1973. 1974. 1974. 1974. 1974. 1975. 1975. 1975.] +trans. cnt: [2250. 3500. 2500. 3000. 4100. 6000. 8000. 4528. 4000. 5000.] ++
+
+
+
+
+
+
+In [10]:
+
+
+
+
+
+apl.eval("""
+year←data[;2]⊣' ⍝ grab the second column and assign'
+transistor_count←data[;1]⊣' ⍝ grab the first column and assign'
+
+⎕←'year: ',10↑year
+⎕←'trans. cnt: ',10↑transistor_count ⋄
+""");
+
+
+
+
+
+
+
+
+
+
+
+year: 1971 1972 1973 1974 1974 1974 1974 1975 1975 1975 +trans. cnt: 2250 3500 2500 3000 4100 6000 8000 4528 4000 5000 ++
+
+
+
+
+
+
+
+
+Goal: solve yi = A × year + B for A, B
+
+
+
+
+
+
+
+In [11]:
+
+
+
+
+
+yi = np.log(transistor_count)
+run("yi← ⍟ transistor_count");
+
+
+
+
+
+
+
+
+
+Use least squares
+
+
+
+
+
+
+
+In [12]:
+
+
+
+
+
+model = np.polynomial.Polynomial.fit(year, yi, deg=1)
+model = model.convert()
+model
+
+
+
+
+
+
+
+
+Out[12]:
+
+$x \mapsto \text{-666.32640635} + \text{0.34163208}\,x$
+
+
+
+
+
+
+
+
+In [13]:
+
+
+
+
+
+run("""
+_←'https://aplcart.info?q=linear%20fit'
+_←'matrix div ⌹ ⎕÷'
+model←yi⌹(1,⍪year)
+(B A)←model
+⎕←'x' ' ' '↦'(1↓9⍕B)'+'(1↓9⍕A)'x'
+""");
+
+
+
+
+
+
+
+
+
+
+
+x ↦ ¯666.326406352 + 0.341632083 x ++
+
+
+
+
+
+
+
+
+The individual parameters A and B are the coefficients of our linear model:
+
+
+
+
+
+
+
+In [14]:
+
+
+
+
+
+B, A = model
+# (B A)←model
+
+
+
+
+
+
+
+
+
+Increase rate = e^(2 × A)
+
+
+
+
+
+
+In [15]:
+
+
+
+
+
+print(f"Rate of semiconductors added on a chip every 2 years: {np.exp(2 * A):.2f}")
+
+
+
+
+
+
+
+
+
+
+
+Rate of semiconductors added on a chip every 2 years: 1.98 ++
+
+
+
+
+
+
+In [16]:
+
+
+
+
+
+run("""
+⎕←'Rate of semiconductors added on a chip every 2 years: ',1↓2⍕*2×A
+""");
+
+
+
+
+
+
+
+
+
+
+
+Rate of semiconductors added on a chip every 2 years: 1.98 ++
+
+
+
+
+
+
+
+
+
+Plot & compare:
+-
+
- Least squares +
- Moore's law +
- Real world data +
+
+
+
+
+
+
+
+
+style sheet: fivethirtyeight
+
+
+
+
+
+
+In [17]:
+
+
+
+
+
+transistor_count_predicted = np.exp(B) * np.exp(A * year)
+transistor_Moores_law = Moores_law(year)
+plt.style.use("fivethirtyeight")
+plt.semilogy(year, transistor_count, "s", label="MOS transistor count")
+plt.semilogy(year, transistor_count_predicted, label="linear regression")
+
+
+plot1 = plt.plot(year, transistor_Moores_law, label="Moore's Law")
+plt.title(
+ "MOS transistor count per microprocessor\n"
+ + "every two years \n"
+ + "Transistor count was x{:.2f} higher".format(np.exp(A * 2))
+)
+plt.xlabel("year introduced")
+plt.legend(loc="center left", bbox_to_anchor=(1, 0.5))
+plt.ylabel("# of transistors\nper microprocessor")
+
+
+
+
+
+
+
+
+Out[17]:
+
+
+Text(0, 0.5, '# of transistors\nper microprocessor')+
+
+
+
+
+
+
+
+
+
+
+
+
+
+In [18]:
+
+
+
+
+
+plot1[0].figure
+
+
+
+
+
+
+
+
+Out[18]:
+
+
+
+
+
+
+
+
+
+
+
+In [19]:
+
+
+
+
+
+run("""
+_←']Chart ⍝ useful, windows only'
+_538←{_sp←⍵
+ _←_sp.SetLineStyles do Causeway.LineStyle.Solid
+ FromHtml←Causeway.ColorTranslator.{FromHtml'#',⍵}
+ _←'FromHtml←{256⊥255,16⊥⍉3 2⍴¯1+⍵⍳⍨⎕D,⎕C ⎕A}'
+ _←'https://github.com/matplotlib/matplotlib/blob/main/lib/matplotlib/mpl-data/stylelib/fivethirtyeight.mplstyle'
+ _←_sp.SetColors do⊂Causeway.{Color.(⍵)}FromHtml¨'008fd5' 'fc4f30' 'e5ae38' '6d904f' '8b8b8b' '810f7c'
+ _←_sp.SetBackground do Causeway.{Color.(⍵)}FromHtml'f0f0f0'
+ _←_sp.SetKeyBackground do Causeway.{Color.(⍵)}FromHtml'f0f0f0'
+ _←_sp.SetAxisStyle do(Causeway.{Color.(⍵)}FromHtml'cbcbcb')Causeway.LineStyle.Solid 0.5
+ _←_sp.SetGridLineStyle do(Causeway.{Color.(⍵)}FromHtml'cbcbcb')Causeway.LineStyle.Solid 0.5
+ _←_sp.SetPenWidths do 3
+ font←'DejaVu Sans'
+ _←_sp.SetHeadingFont do font
+ _←_sp.SetCaptionFont do font
+ _←_sp.SetLabelFont do font
+ _←_sp.SetKeyFont do font
+ _sp.KeyStyle←Causeway.KeyStyles.(Boxed+Vertical+MiddleAlign)
+ _sp.XAxisStyle←Causeway.XAxisStyles.CenteredCaption
+ _sp.YAxisStyle←Causeway.YAxisStyles.CenteredCaption
+ _sp.YLabelFormat←'0.0E00'
+ _sp.ScatterPlotStyle←Causeway.ScatterPlotStyles.GridLines
+ _sp.LineGraphStyle←Causeway.LineGraphStyles.GridLines
+ _sp}
+do←{⍎'⍺⍺ ⍵ ⋄ 0' ⋄ ⍺⍺}
+'InitCauseway'⎕CY'sharpplot'
+InitCauseway ⍬
+sp←⎕NEW Causeway.SharpPlot
+""");
+
+
+
+
+
+
+
+In [20]:
+
+
+
+
+
+import IPython
+IPython.display.SVG(run("""
+transistor_count_predicted←(*B)×(*A×year)
+transistor_Moores_law←Moores_law year
+sp←_538 sp
+_←sp.SetKeyText do'MOS transistor count' 'linear regression' 'Moore''s Law'
+sp.Heading←'MOS transistor count per microprocessor',(⎕UCS 10),'every two years',(⎕UCS 10),'Transistor count was x ',(1↓2⍕*A×2),' higher'
+sp.XCaption←'year introduced'
+sp.YCaption←'# of transistors',(⎕UCS 10),'per microprocessor'
+sp.YAxisStyle←sp.YAxisStyle+Causeway.YAxisStyles.LogScale
+sp.KeyStyle←sp.KeyStyle+Causeway.KeyStyles.RightAlign
+_←sp.SetMarkers do Causeway.Marker.Block
+_←sp.SetMargins do 80 30 50 150
+_←sp.SetXTickMarks do 10
+_←sp.SetYTickMarks do 2
+_←sp.DrawScatterPlot do(transistor_count)year
+_←sp.DrawLineGraph do transistor_count_predicted year
+_←sp.DrawLineGraph do transistor_Moores_law year
+sp.RenderSvg ⍬
+"""))
+
+
+
+
+
+
+
+
+Out[20]:
+
+
+
+
+
+
+
+
+
+
+
+In [21]:
+
+
+
+
+
+IPython.display.SVG(run("sp.RenderSvg ⍬"))
+
+
+
+
+
+
+
+
+Out[21]:
+
+
+
+
+
+
+
+
+
+
+
+
+
+Zoom in on year == 2017, compare:
-
+
- Moore's law +
- Average transistor count +
- Real world data +
Use alpha=0.2, opaque points mean overlap
+
+
+
+
+
+
+In [22]:
+
+
+
+
+
+transistor_count2017 = transistor_count[year == 2017]
+print(
+ transistor_count2017.max(), transistor_count2017.min(), transistor_count2017.mean()
+)
+y = np.linspace(2016.5, 2017.5)
+your_model2017 = np.exp(B) * np.exp(A * y)
+Moore_Model2017 = Moores_law(y)
+
+
+
+
+
+
+
+
+
+
+
+19200000000.0 250000000.0 7050000000.0 ++
+
+
+
+
+
+
+In [23]:
+
+
+
+
+
+run("""
+transistor_count2017←transistor_count[⍸year=2017]
+_←'transistor_count2017←(year=2017)/transistor_count'
+mean←{v←⍵ ⋄ (+/v)÷(≢v)}
+⎕←(⌈/transistor_count2017)(⌊/transistor_count2017)(mean transistor_count2017)
+
+y←2016.5 2017.5
+your_model2017←(*B)×(*A×y)
+Moore_Model2017←Moores_law y
+""");
+
+
+
+
+
+
+
+
+
+
+
+1.92E10 250000000 7050000000 ++
+
+
+
+
+
+
+In [24]:
+
+
+
+
+
+plt.plot(
+ 2017 * np.ones(np.sum(year == 2017)),
+ transistor_count2017,
+ "ro",
+ label="2017",
+ alpha=0.2,
+)
+plt.plot(2017, transistor_count2017.mean(), "g+", markersize=20, mew=6)
+
+plt.plot(y, your_model2017, label="Your prediction")
+plot2 = plt.plot(y, Moore_Model2017, label="Moores law")
+plt.ylabel("# of transistors\nper microprocessor")
+plt.legend()
+
+
+
+
+
+
+
+
+Out[24]:
+
+
+<matplotlib.legend.Legend at 0x7f804028cc80>+
+
+
+
+
+
+
+
+
+
+
+
+
+
+In [25]:
+
+
+
+
+
+plot2[0].figure
+
+
+
+
+
+
+
+
+Out[25]:
+
+
+
+
+
+
+
+
+
+
+
+In [26]:
+
+
+
+
+
+IPython.display.SVG(run("""
+_←sp.Reset do ⍬
+sp←_538 sp
+_←sp.SetXRange do y
+_←sp.SetYRange do⍎'2.4e10'
+_←sp.SetXTickMarks do 0.2
+_←sp.SetMarkers do⊂Causeway.Marker.(Dot Plus)
+_←sp.SetMarkerColors do Causeway.Color.LightCoral
+_←sp.SetMarkerScales do 6
+_←sp.SetMargins do 5 15 55 20
+_←sp.SetYTickMarks do⍎'0.5e10'
+sp.YCaption←'# of transistors',(⎕UCS 10),'per microprocessor'
+_←sp.DrawScatterPlot do(transistor_count2017)(2017×(+/year=2017)⍴1)
+_←sp.SetMarkerColors do Causeway.Color.Green
+_←sp.SetPenWidths do 4
+_←sp.SetMarkerScales do 3
+_←sp.SetKeyText do'2017' ' ' 'Your prediction' 'Moore''s Law'
+_←sp.DrawScatterPlot do,¨(mean transistor_count2017)2017
+sp←_538 sp
+sp.KeyStyle←sp.KeyStyle+Causeway.KeyStyles.BottomAlign
+_←sp.DrawLineGraph do your_model2017 y
+_←sp.DrawLineGraph do Moore_Model2017 y
+sp.RenderSvg ⍬
+"""))
+
+
+
+
+
+
+
+
+Out[26]:
+
+
+
+
+
+
+
+
+
+
+
+
+In [27]:
+
+
+
+
+
+IPython.display.SVG(run("sp.RenderSvg ⍬"))
+
+
+
+
+
+
+
+
+Out[27]:
+
+
+
+
+
+
+
+
+
+
+
+
+
+-
+
- Least squares: close to mean +
- Moore's law: close to max +
+
+
+
+
+
+
+
+
+-
+
- Least squares: close to mean +
- Moore's law: close to max +
+
+
+
+
+
+
+
+
+Share as zipped arrays and CSV¶
-
+
np.savez: NumPy arrays for other Python sessions
+- APL namespace (also possible: component file) +
np.savetxt,⎕CSV: CSV
+
savez with notes=notes
+
+
+
+
+
+
+In [28]:
+
+
+
+
+
+notes = "the arrays in this file are the result of a linear regression model\n"
+notes += "the arrays include\nyear: year of manufacture\n"
+notes += "transistor_count: number of transistors reported by manufacturers in a given year\n"
+notes += "transistor_count_predicted: linear regression model = exp({:.2f})*exp({:.2f}*year)\n".format(
+ B, A
+)
+notes += "transistor_Moores_law: Moores law =exp({:.2f})*exp({:.2f}*year)\n".format(
+ B_M, A_M
+)
+notes += "regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B"
+print(notes)
+
+
+
+
+
+
+
+
+
+
+
+the arrays in this file are the result of a linear regression model +the arrays include +year: year of manufacture +transistor_count: number of transistors reported by manufacturers in a given year +transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year) +transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year) +regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B ++
+
+
+
+
+
+
+In [29]:
+
+
+
+
+
+run("""
+nl←⎕UCS 10
+notes←'the arrays in this file are the result of a linear regression model',nl
+notes,←'the arrays include',nl,'year: year of manufacture',nl
+notes,←'transistor_count: number of transistors reported by manufacturers in a given year',nl
+notes,←'transistor_count_predicted: linear regression model = exp(',(1↓2⍕B),')*exp(',(1↓2⍕A),'*year)',nl
+
+notes,←'transistor_Moores_law: Moores law =exp(',(1↓2⍕B_M),')*exp(',(1↓2⍕A_M),'*year)',nl
+
+notes,←'regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B'
+⎕←notes
+""");
+
+
+
+
+
+
+
+
+
+
+
+the arrays in this file are the result of a linear regression model +the arrays include +year: year of manufacture +transistor_count: number of transistors reported by manufacturers in a given year +transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year) +transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year) +regression_csts: linear regression constants A and B for log(transistor_count)=A*year+B ++
+
+
+
+
+
+
+In [30]:
+
+
+
+
+
+np.savez(
+ "mooreslaw_regression.npz",
+ notes=notes,
+ year=year,
+ transistor_count=transistor_count,
+ transistor_count_predicted=transistor_count_predicted,
+ transistor_Moores_law=transistor_Moores_law,
+ regression_csts=(A, B),
+)
+
+
+
+
+
+
+
+In [31]:
+
+
+
+
+
+run("""
+_ns←⎕NS'notes' 'year' 'transistor_count' 'transistor_count_predicted' 'transistor_Moores_law'
+_ns.regression_csts←A B
+_←'https://dyalog.github.io/link/4.0/API/Link.Fix'
+_←(⊂⎕SE.Dyalog.Array.Serialise _ns)⎕NPUT'mooreslaw_regression.apln' 1
+
+_←'"" "mooreslaw_regression"⎕SE.Link.Fix ⎕SE.Dyalog.Array.Serialise _ns'
+""");
+
+
+
+
+
+
+
+In [32]:
+
+
+
+
+
+results = np.load("mooreslaw_regression.npz")
+run("""
+results←⎕SE.Dyalog.Array.Deserialise⊃⎕NGET'mooreslaw_regression.apln'
+""");
+
+
+
+
+
+
+
+In [33]:
+
+
+
+
+
+print( results["regression_csts"][1])
+run("⎕←1↓13⍕results. regression_csts [2]");
+
+
+
+
+
+
+
+
+
+
+
+-666.3264063536233 +¯666.3264063521225 ++
+
+
+
+
+
+
+In [34]:
+
+
+
+
+
+! ls
+
+
+
+
+
+
+
+
+
+
+
+air-quality-data.csv +mooreslaw_regression.apln +mooreslaw_regression.csv +mooreslaw_regression.npz +mooreslaw-tutorial.ipynb +mooreslaw-tutorial.md +mooreslaw-tutorial.slides.html +pairing.md +quadna.dws +r2 +save-load-arrays.md +_static +text_preprocessing.py +transistor_data.csv +tutorial-air-quality-analysis.md +tutorial-deep-learning-on-mnist.md +tutorial-deep-reinforcement-learning-with-pong-from-pixels.md +tutorial-ma.md +tutorial-nlp-from-scratch +tutorial-nlp-from-scratch.md +tutorial-plotting-fractals +tutorial-plotting-fractals.md +tutorial-static_equilibrium.md +tutorial-style-guide.md +tutorial-svd.md +tutorial-x-ray-image-processing +tutorial-x-ray-image-processing.md +who_covid_19_sit_rep_time_series.csv ++
+
+
+
+
+
+
+In [35]:
+
+
+
+
+
+run("⎕←↑⎕SH'ls'");
+
+
+
+
+
+
+
+
+
+
+
+air-quality-data.csv +mooreslaw_regression.apln +mooreslaw_regression.csv +mooreslaw_regression.npz +mooreslaw-tutorial.ipynb +mooreslaw-tutorial.md +mooreslaw-tutorial.slides.html +pairing.md +quadna.dws +r2 +save-load-arrays.md +_static +text_preprocessing.py +transistor_data.csv +tutorial-air-quality-analysis.md +tutorial-deep-learning-on-mnist.md +tutorial-deep-reinforcement-learning-with-pong-from-pixels.md +tutorial-ma.md +tutorial-nlp-from-scratch +tutorial-nlp-from-scratch.md +tutorial-plotting-fractals +tutorial-plotting-fractals.md +tutorial-static_equilibrium.md +tutorial-style-guide.md +tutorial-svd.md +tutorial-x-ray-image-processing +tutorial-x-ray-image-processing.md +who_covid_19_sit_rep_time_series.csv ++
+
+
+
+
+
+
+
+
+CSV¶
-
+
np.savetxtwithheader=head
+- need 2D array +
+
+
+
+
+
+
+In [36]:
+
+
+
+
+
+head = "the columns in this file are the result of a linear regression model\n"
+head += "the columns include\nyear: year of manufacture\n"
+head += "transistor_count: number of transistors reported by manufacturers in a given year\n"
+head += "transistor_count_predicted: linear regression model = exp({:.2f})*exp({:.2f}*year)\n".format(
+ B, A
+)
+head += "transistor_Moores_law: Moores law =exp({:.2f})*exp({:.2f}*year)\n".format(
+ B_M, A_M
+)
+head += "year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:"
+print(head)
+
+
+
+
+
+
+
+
+
+
+
+the columns in this file are the result of a linear regression model +the columns include +year: year of manufacture +transistor_count: number of transistors reported by manufacturers in a given year +transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year) +transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year) +year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law: ++
+
+
+
+
+
+
+In [37]:
+
+
+
+
+
+run("""
+head←'the columns in this file are the result of a linear regression model',nl
+head,←'the columns include',nl,'year: year of manufacture',nl
+head,←'transistor_count: number of transistors reported by manufacturers in a given year',nl
+head,←'transistor_count_predicted: linear regression model = exp(',(1↓2⍕B),')*exp(',(1↓2⍕A),'*year)',nl
+
+head,←'transistor_Moores_law: Moores law =exp(',(1↓2⍕B_M),')*exp(',(1↓2⍕A_M),'*year)',nl
+
+head,←'year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law:'
+⎕←head
+""");
+
+
+
+
+
+
+
+
+
+
+
+the columns in this file are the result of a linear regression model +the columns include +year: year of manufacture +transistor_count: number of transistors reported by manufacturers in a given year +transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year) +transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year) +year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law: ++
+
+
+
+
+
+
+
+
+Build 2D array by np.blocking 1D vectors with np.newaxis (1-col arrays) together
+
+
+
+
+
+
+In [38]:
+
+
+
+
+
+print( year.shape)
+run("⎕←⍴year");
+
+print( year[:,np.newaxis].shape)
+run("⎕←⍴⍪year");
+
+
+
+
+
+
+
+
+
+
+
+(179,) +179 +(179, 1) +179 1 ++
+
+
+
+
+
+
+In [39]:
+
+
+
+
+
+output = np.block(
+ [
+ year[:, np.newaxis],
+ transistor_count[:, np.newaxis],
+ transistor_count_predicted[:, np.newaxis],
+ transistor_Moores_law[:, np.newaxis],
+ ]
+)
+
+
+
+
+
+
+
+In [40]:
+
+
+
+
+
+run("""
+output←year,transistor_count,transistor_count_predicted,⍪transistor_Moores_law
+""");
+
+
+
+
+
+
+
+In [41]:
+
+
+
+
+
+np.savetxt("mooreslaw_regression.csv", X=output, delimiter=",", header=head)
+
+
+
+
+
+
+
+In [42]:
+
+
+
+
+
+! head mooreslaw_regression.csv
+
+
+
+
+
+
+
+
+
+
+
+# the columns in this file are the result of a linear regression model +# the columns include +# year: year of manufacture +# transistor_count: number of transistors reported by manufacturers in a given year +# transistor_count_predicted: linear regression model = exp(-666.33)*exp(0.34*year) +# transistor_Moores_law: Moores law =exp(-675.38)*exp(0.35*year) +# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law: +1.971000000000000000e+03,2.250000000000000000e+03,1.130514785642591733e+03,2.249999999999916326e+03 +1.972000000000000000e+03,3.500000000000000000e+03,1.590908400344571419e+03,3.181980515339620069e+03 +1.973000000000000000e+03,2.500000000000000000e+03,2.238793840142739555e+03,4.500000000000097316e+03 ++
+
+
+
+
+
+
+In [43]:
+
+
+
+
+
+run("""
+content←⊂(('^'⎕R'# &')head),nl,output(⎕CSV⍠'Separator' ',')''
+_←content ⎕NPUT'mooreslaw_regression.csv' 1
+""");
+
+
+
+
+
+
+
+In [44]:
+
+
+
+
+
+run("""
+⎕←↑'UTF-8'∘⎕UCS¨⎕UCS¨⎕SH'head mooreslaw_regression.csv'
+⎕←''
+⎕←↑⎕SH'head mooreslaw_regression.csv'
+""");
+
+
+
+
+
+
+
+
+
+
+
+# the columns in this file are the result of a linear regression model +# the columns include +# year: year of manufacture +# transistor_count: number of transistors reported by manufacturers in a given year +# transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year) +# transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year) +# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law: +1971,2250,1130.514785668682,2249.999999999916 +1972,3500,1590.908400380202,3181.98051533962 +1973,2500,2238.793840191098,4500.000000000097 + +# the columns in this file are the result of a linear regression model +# the columns include +# year: year of manufacture +# transistor_count: number of transistors reported by manufacturers in a given year +# transistor_count_predicted: linear regression model = exp(¯666.33)*exp(0.34*year) +# transistor_Moores_law: Moores law =exp(¯675.38)*exp(0.35*year) +# year:, transistor_count:, transistor_count_predicted:, transistor_Moores_law: +1971,2250,1130.514785668682,2249.999999999916 +1972,3500,1590.908400380202,3181.98051533962 +1973,2500,2238.793840191098,4500.000000000097 ++
+
+
+
+
+
+
+
+
+
+
+
+Functions we used¶
+| + | + | + |
|---|---|---|
np.loadtxt |
+←→ | +⎕CSV [;2 3] 1↓ |
+
np.log |
+←→ | +⍟ |
+
np.exp |
+←→ | +* |
+
lambda |
+←→ | +{⍵} |
+
plt.semilogy |
+←→ | +Causeway.YAxisStyles.LogScale |
+
plt.plot |
+←→ | +sp.RenderSvg, sp.Draw* (OOP) |
+
x[:10] |
+←→ | +10↑x |
+
x[year == 2017] |
+←→ | +(year=2017)/x or x[⍸year=2017] |
+
np.block |
+←→ | +, ⍪ |
+
np.newaxis |
+←→ | +⍪ |
+
np.savez |
+←→ | +⎕SE.Dyalog.Array.Serialise ⎕NPUT |
+
Some points¶
-
+
- Specific functions ←→ small composable functions (FP) +
- Composing symbols:
⍟ ○*⌹ ⎕÷
+ - Nested function calls ←→ shallow definitions (debugging context) +
- Mature ecosystem (plot styles, exports) ←→ workarounds/hacks/rolling your own +
+
+
+
+
+
+
+
+
+Some points¶
-
+
- Specific functions ←→ small composable functions (FP) +
- Composing symbols:
⍟ ○*⌹ ⎕÷
+ - Nested function calls ←→ shallow definitions (debugging context) +
- Mature ecosystem (plot styles, exports) ←→ workarounds/hacks/rolling your own +
+
+
+
+
+
+
+
+
+Chat, Q&A¶
happy little demos?
+
+
+
+
+
+
+
+
+
+
+-
+
- Point free appendix +
- 1-liner glass cleaner fractal +
- Open emacs from RIDE, prefix completion, d.apln jump +
- XML: 1-liner ]Defs workflow, https://apl.quest vids +
@MaxCan-Code "Max Sun APL"
+
+
+
+
+
+
+
+
+
+Point-free (tacit) demo¶
-
+
- Opinionated takes +
- Not liable for mental dmg +
+
+
+
+
+
+
+
+
+-
+
n+2letter name/notation/definition/formula/src code?
+- see as 1 unit (
poly morph ism←→polymorphism):-
+
(B_M×⍨∘*⍨∘*A_M×⊢)
+
+ - vs autocomplete +
+
+
+
+
+
+
+In [45]:
+
+
+
+
+
+Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)
+Moores_law(1971), Moores_law(1973)
+
+
+
+
+
+
+
+
+Out[45]:
+
+
+(np.float64(2249.9999999999163), np.float64(4500.000000000097))+
+
+
+
+
+
+
+In [46]:
+
+
+
+
+
+run("""
+_←'Moores_law = lambda year: np.exp(B_M) * np.exp(A_M * year)'
+ Moores_law ← { year←⍵ ⋄ (* B_M) × *(A_M × year)}
+⎕← (B_M ×⍨∘*⍨∘ * A_M × ⊢)1971 1973
+_←' └────────┘'
+
+⎕←(B_M×⍨∘*⍨∘*A_M×⊢)1971 1973
+⎕←Moores_law 1971 1973
+
+_←'https://aplcart.info?q=split%20compose (looks better next ver)'
+""");
+
+
+
+
+
+
+
+
+
+
+
+2250 4500 +2250 4500 +2250 4500 ++
+
+
+
+
+
+
+
+
+-
+
n+2letter name/notation/definition/formula/src code?
+- see as 1 unit (
poly morph ism←→polymorphism):-
+
(⊣⌹1,∘⍪⊢)
+
+ - vs autocomplete +
+
+
+
+
+
+
+In [47]:
+
+
+
+
+
+model = np.polynomial.Polynomial.fit(year, yi, deg=1)
+model = model.convert()
+B, A = model
+print(B ,A)
+
+
+
+
+
+
+
+
+
+
+
+-666.3264063536233 0.3416320825591619 ++
+
+
+
+
+
+
+In [48]:
+
+
+
+
+
+run("""
+⎕←yi⌹(1,⍪year)
+
+⎕←yi ⌹1, ⍪ year
+⎕←yi(⊣⌹1,∘⍪⊢)year
+""");
+
+
+
+
+
+
+
+
+
+
+
+¯666.3264064 0.3416320826 +¯666.3264064 0.3416320826 +¯666.3264064 0.3416320826 ++
+
+
+
+
+
+
+
+
+-
+
n+2letter name/notation/definition/formula/src code?
+- see as 1 unit (
poly morph ism←→polymorphism):-
+
(⌈/ , ⌊/ , +/÷≢)
+
+ - vs autocomplete +
+
+
+
+
+
+
+In [49]:
+
+
+
+
+
+print(
+ transistor_count2017.max(), transistor_count2017.min(), transistor_count2017.mean()
+)
+
+
+
+
+
+
+
+
+
+
+
+19200000000.0 250000000.0 7050000000.0 ++
+
+
+
+
+
+
+In [50]:
+
+
+
+
+
+run("""
+mean←{v←⍵ ⋄ (+/v)÷(≢v)}
+⎕←(⌈/transistor_count2017)(⌊/transistor_count2017)(mean transistor_count2017)
+⎕←(⌈/ , ⌊/ , +/÷≢) transistor_count2017
+
+⎕←(⌈/,⌊/,+/÷≢)transistor_count2017
+""");
+
+
+
+
+
+
+
+
+
+
+
+1.92E10 250000000 7050000000 +1.92E10 250000000 7050000000 +1.92E10 250000000 7050000000 ++
+
+
+
+
+
+
+
+
+
+
+
+Related Work¶
-
+
- Alan J. Perlis. 1977. In praise of APL: a language for lyrical programming. SIGAPL APL Quote Quad 8, 2 (December 1977), 44–47. https://doi.org/10.1145/586015.586019 +
J: primary parts of speech:
+
| + | + | + |
|---|---|---|
| noun | +←→ | +data | +
| verb | +←→ | +function on nouns | +
| adverb | +←→ | +modify verb | +
https://code.jsoftware.com/wiki/Vocabulary/Words#Parts_Of_Speech
+
+
+
+
+
+
+
+
+
+
+Features¶
+
+
+
+
+
+
+
+
+
+
+Edit in-place
+
+
+
+
+
+
+
+
+
+
+Rich text (Markdown), run code, embed results, transitions, zoom
+
+
+
+
+
+
+
+
+
+
+Interactive, literate (Donald Knuth)
+
+
+
+
+
+
+
+
+
+
+
+
+Install: JupterLab
+-
+
- extension: jupyterlab-rise +
- View -> Command Palette -> Open Reveal +
(Py'n'APL: docs)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Wrap up¶
-
+
- Features:
-
+
- Edit in-place +
- Markdown text +
- Run code +
+ - Tools:
-
+
- Py'n'APL +
- SharpPlot +
- JupyterLab + jupyterlab-rise +
- Dyalog Jupyter kernel + APL language bar +
- Jupytext +
+
Slides and source: https://github.com/MaxCan-Code/numpy-tutorials
+"Max Sun APL"
+-
+
- Making documents +
- Discuss literate programming +
Questions/Comments/Chat¶
+
+
+
+
+
+
+
+
+
+Extras¶
Markdown syntax
+https://MaxCan-Code.github.io/numpy-tutorials/content/mooreslaw-tutorial.slides.html
+-
+
- nbconv template: word-wrap, c/t +