summaryrefslogtreecommitdiff
path: root/README_VIM9.md
diff options
context:
space:
mode:
authorBram Moolenaar <Bram@vim.org>2020-01-26 15:56:19 +0100
committerBram Moolenaar <Bram@vim.org>2020-01-26 15:56:19 +0100
commit8a7d6542b33e5d2b352262305c3bfdb2d14e1cf8 (patch)
tree8e5f241129a1c690ea81d697a72fb4c1704c0cb6 /README_VIM9.md
parent1d9215b9aaa120b9d78fee49488556f73007ce78 (diff)
downloadvim-git-8a7d6542b33e5d2b352262305c3bfdb2d14e1cf8.tar.gz
patch 8.2.0149: maintaining a Vim9 branch separately is more workv8.2.0149
Problem: Maintaining a Vim9 branch separately is more work. Solution: Merge the Vim9 script changes.
Diffstat (limited to 'README_VIM9.md')
-rw-r--r--README_VIM9.md344
1 files changed, 344 insertions, 0 deletions
diff --git a/README_VIM9.md b/README_VIM9.md
new file mode 100644
index 000000000..b77d013c0
--- /dev/null
+++ b/README_VIM9.md
@@ -0,0 +1,344 @@
+![Vim Logo](https://github.com/vim/vim/blob/master/runtime/vimlogo.gif)
+
+# What is Vim9?
+
+This is an experimental side of [Vim](https://github.com/vim/vim).
+It explores ways of making Vim script faster and better.
+
+WARNING: The Vim9 script features are in the early stages of development,
+anything can break!
+
+# Why Vim9?
+
+## 1. FASTER VIM SCRIPT
+
+The third item on the poll results of 2018, after popup windows and text
+properties, is faster Vim script. So how do we do that?
+
+I have been throwing some ideas around, and soon came to the conclusion
+that the current way functions are called and executed, with
+dictionaries for the arguments and local variables, is never going to be
+very fast. We're lucky if we can make it twice as fast. The overhead
+of a function call and executing every line is just too high.
+
+So what then? We can only make something fast by having a new way of
+defining a function, with similar but different properties of the old
+way:
+* Arguments are only available by name, not through the a: dictionary or
+ the a:000 list.
+* Local variables are not available in an l: dictionary.
+* A few more things that slow us down, such as exception handling details.
+
+I Implemented a "proof of concept" and measured the time to run a simple
+for loop with an addition (Justin used this example in his presentation,
+full code is below):
+
+``` vim
+ let sum = 0
+ for i in range(1, 2999999)
+ let sum += i
+ endfor
+```
+
+| how | time in sec |
+| --------| -------- |
+| Vim old | 5.018541 |
+| Python | 0.369598 |
+| Lua | 0.078817 |
+| Vim new | 0.073595 |
+
+That looks very promising! It's just one example, but it shows how much
+we can gain, and also that Vim script can be faster than builtin
+interfaces.
+
+In practice the script would not do something useless as counting but change
+the text. For example, re-indent all the lines:
+
+``` vim
+ let totallen = 0
+ for i in range(1, 100000)
+ call setline(i, ' ' .. getline(i))
+ let totallen += len(getline(i))
+ endfor
+```
+
+| how | time in sec |
+| --------| -------- |
+| Vim old | 0.853752 |
+| Python | 0.304584 |
+| Lua | 0.286573 |
+| Vim new | 0.190276 |
+
+The differences are smaller, but Vim 9 script is clearly the fastest.
+
+How does Vim9 script work? The function is first compiled into a sequence of
+instructions. Each instruction has one or two parameters and a stack is
+used to store intermediate results. Local variables are also on the
+stack, space is reserved during compilation. This is a fairly normal
+way of compilation into an intermediate format, specialized for Vim,
+e.g. each stack item is a typeval_T. And one of the instructions is
+"execute Ex command", for commands that are not compiled.
+
+
+## 2. PHASING OUT INTERFACES
+
+Attempts have been made to implement functionality with built-in script
+languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much
+foothold, for various reasons.
+
+Instead of using script language support in Vim:
+* Encourage implementing external tools in any language and communicate
+ with them. The job and channel support already makes this possible.
+ Really any language can be used, also Java and Go, which are not
+ available built-in.
+* Phase out the built-in language interfaces, make maintenance a bit easier
+ and executables easier to build. They will be kept for backwards
+ compatibility, no new features.
+* Improve the Vim script language, it is used to communicate with the external
+ tool and implements the Vim side of the interface. Also, it can be used when
+ an external tool is undesired.
+
+All together this creates a clear situation: Vim with the +eval feature
+will be sufficient for most plugins, while some plugins require
+installing a tool that can be written in any language. No confusion
+about having Vim but the plugin not working because some specific
+language is missing. This is a good long term goal.
+
+Rationale: Why is it better to run a tool separately from Vim than using a
+built-in interface and interpreter? Take for example something that is
+written in Python:
+* The built-in interface uses the embedded python interpreter. This is less
+ well maintained than the python command. Building Vim with it requires
+ installing developer packages. If loaded dynamically there can be a version
+ mismatch.
+* When running the tool externally the standard python command can be used,
+ which is quite often available by default or can be easily installed.
+* The built-in interface has an API that is unique for Vim with Python. This is
+ an extra API to learn.
+* A .py file can be compiled into a .pyc file and execute much faster.
+* Inside Vim multi-threading can cause problems, since the Vim core is single
+ threaded. In an external tool there are no such problems.
+* The Vim part is written in .vim files, the Python part is in .py files, this
+ is nicely separated.
+* Disadvantage: An interface needs to be made between Vim and Python.
+ JSON is available for this, and it's fairly easy to use. But it still
+ requires implementing asynchronous communication.
+
+
+## 3. BETTER VIM SCRIPT
+
+To make Vim faster a new way of defining a function needs to be added.
+While we are doing that, since the lines in this function won't be fully
+backwards compatible anyway, we can also make Vim script easier to use.
+In other words: "less weird". Making it work more like modern
+programming languages will help. No surprises.
+
+A good example is how in a function the arguments are prefixed with
+"a:". No other language I know does that, so let's drop it.
+
+Taking this one step further is also dropping "s:" for script-local variables;
+everything at the script level is script-local by default. Since this is not
+backwards compatible it requires a new script style: Vim9 script!
+
+It should be possible to convert code from other languages to Vim
+script. We can add functionality to make this easier. This still needs
+to be discussed, but we can consider adding type checking and a simple
+form of classes. If you look at JavaScript for example, it has gone
+through these stages over time, adding real class support and now
+TypeScript adds type checking. But we'll have to see how much of that
+we actually want to include in Vim script. Ideally a conversion tool
+can take Python, JavaScript or TypeScript code and convert it to Vim
+script, with only some things that cannot be converted.
+
+Vim script won't work the same as any specific language, but we can use
+mechanisms that are commonly known, ideally with the same syntax. One
+thing I have been thinking of is assignments without ":let". I often
+make that mistake (after writing JavaScript especially). I think it is
+possible, if we make local variables shadow commands. That should be OK,
+if you shadow a command you want to use, just rename the variable.
+Using "let" and "const" to declare a variable, like in JavaScript and
+TypeScript, can work:
+
+
+``` vim
+def MyFunction(arg: number): number
+ let local = 1
+ let todo = arg
+ const ADD = 88
+ while todo > 0
+ local += ADD
+ --todo
+ endwhile
+ return local
+enddef
+```
+
+The similarity with JavaScript/TypeScript can also be used for dependencies
+between files. Vim currently uses the `:source` command, which has several
+disadvantages:
+* In the sourced script, is not clear what it provides. By default all
+ functions are global and can be used elsewhere.
+* In a script that sources other scripts, it is not clear what function comes
+ from what sourced script. Finding the implementation is a hassle.
+* Prevention of loading the whole script twice must be manually implemented.
+
+We can use the `:import` and `:export` commands from the JavaScript standard to
+make this much better. For example, in script "myfunction.vim" define a
+function and export it:
+
+``` vim
+vim9script " Vim9 script syntax used here
+
+let local = 'local variable is not exported, script-local'
+
+export def MyFunction() " exported function
+...
+
+def LocalFunction() " not exported, script-local
+...
+```
+
+And in another script import the function:
+
+``` vim
+vim9script " Vim9 script syntax used here
+
+import MyFunction from 'myfunction.vim'
+```
+
+This looks like JavaScript/TypeScript, thus many users will understand the
+syntax.
+
+These are ideas, this will take time to design, discuss and implement.
+Eventually this will lead to Vim 9!
+
+
+## Code for sum time measurements
+
+Vim was build with -O2.
+
+``` vim
+func VimOld()
+ let sum = 0
+ for i in range(1, 2999999)
+ let sum += i
+ endfor
+ return sum
+endfunc
+
+func Python()
+ py3 << END
+sum = 0
+for i in range(1, 3000000):
+ sum += i
+END
+ return py3eval('sum')
+endfunc
+
+func Lua()
+ lua << END
+ sum = 0
+ for i = 1, 2999999 do
+ sum = sum + i
+ end
+END
+ return luaeval('sum')
+endfunc
+
+def VimNew()
+ let sum = 0
+ for i in range(1, 2999999)
+ let sum += i
+ endfor
+ return sum
+enddef
+
+let start = reltime()
+echo VimOld()
+echo 'Vim old: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo Python()
+echo 'Python: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo Lua()
+echo 'Lua: ' .. reltimestr(reltime(start))
+
+let start = reltime()
+echo VimNew()
+echo 'Vim new: ' .. reltimestr(reltime(start))
+```
+
+## Code for indent time measurements
+
+``` vim
+def VimNew(): number
+ let totallen = 0
+ for i in range(1, 100000)
+ setline(i, ' ' .. getline(i))
+ totallen += len(getline(i))
+ endfor
+ return totallen
+enddef
+
+func VimOld()
+ let totallen = 0
+ for i in range(1, 100000)
+ call setline(i, ' ' .. getline(i))
+ let totallen += len(getline(i))
+ endfor
+ return totallen
+endfunc
+
+func Lua()
+ lua << END
+ b = vim.buffer()
+ totallen = 0
+ for i = 1, 100000 do
+ b[i] = " " .. b[i]
+ totallen = totallen + string.len(b[i])
+ end
+END
+ return luaeval('totallen')
+endfunc
+
+func Python()
+ py3 << END
+cb = vim.current.buffer
+totallen = 0
+for i in range(0, 100000):
+ cb[i] = ' ' + cb[i]
+ totallen += len(cb[i])
+END
+ return py3eval('totallen')
+endfunc
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo VimOld()
+echo 'Vim old: ' .. reltimestr(reltime(start))
+bwipe!
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo Python()
+echo 'Python: ' .. reltimestr(reltime(start))
+bwipe!
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo Lua()
+echo 'Lua: ' .. reltimestr(reltime(start))
+bwipe!
+
+new
+call setline(1, range(100000))
+let start = reltime()
+echo VimNew()
+echo 'Vim new: ' .. reltimestr(reltime(start))
+bwipe!
+```