The-Spiral-Language

useful for people other than myself and get people to sponsor my work on it. Here is the value proposition of Spiral. It is a high-level functional PL that has some features that other languages don't, but that isn't really the point. On mainstream devices like the x86 ones there are a lot of programming languages that are good, and it would be tedious to use Spiral to compile to such platforms compared to using such languages directly. It is a bit how ReasonML compiles to JS. Back when I tried it I found using Typescript easier to deal with. So that is not where I'd like to go into, though using Spiral would have benefits in certain areas. Rather, while reading the [CNX blog](https://www.cnx-software.com/) I realized that while consumer facing AI chips are not here yet, there is a lot of hardware development in the embedded space. They are heterogenous architecture. They have GPU and TPUs in addition to CPUs. And these cross platform interactions within the same system is something that existing languages are really poor at tackling. If you look at Python or C#, for example, you can't really program the GPU on them directly. They are CPU focused, and don't have the right semantics and would be too inefficient to program devices like GPUs directly. The way I've designed Spiral is that you can program the CPU and the GPU and whatever else from within the same language. It is not suitable for just GPUs, check this [demo out](https://github.com/mrakgr/PIM-Programming-In-Spiral-UPMEM-Demo). I recently did a backend for UPMEM devices, which are the first commercialized Process-In-Memory chips. I've posted the link to this on HN yesterday and on the Reddit embedded sub, but I got zero interest. And this is really a pity because that map kernel I've demoed is actually a big deal. Back when I first started working on Spiral, it took me 1.5 years of full time work to get to the point where I could write a program like that in the language. And without backend nesting of the kind that Spiral offers, it is impossible to write those kinds of programs no matter how skilled one is as a programmer. The kind of backend nesting I've demonstrated is not something you can do in F#, Python or any of the languages that I know of. I could easily create such backends for many kinds of hardware. And people would benefit from that because unlike the mainstream computing devices, the hardware coming down the pipeline will have poor language support, nothing on the level of what Spiral offers. For the kinds of heterogeneous architectures I am envisioning, the language designs that are good in the CPU-dominant era, will simply not be suited in the heterogeneous era. I need chances to demonstrate how good Spiral is, but I am not sure how to get them. If I do not get them, the future of computing will be a lot worse off. I wasn't there when Cuda was incumbent so I missed the boat on that, but I'd like it if Spiral became dominant on future computing devices. Not because I was the one who made the language, but simply because no other design is as suited for them.

k by retaining the structure of say solve_linear_system(). I'm not quite sure. But the problem domains they tackle are very similar to this (numerical code). TensorFlow also has two stages of computation -- you define a graph of objects in imperative Python, using Python operator overloading in the same way that LLVMCode in this example does. And then later the separate TensorFlow language is evaluated in parallel / distributed on GPUs. In one case you're defining an SSA graph of ops in LLVM; in the other case you're defining a graph of TensorFlow ops. Note that the semantics of TensorFlow language are distinct from the semantics of Python, e.g. https://ai.google/research/pubs/pub46196 One interesting language in the same vein is Spiral, with "first-class staging" for machine learning: https://github.com/mrakgr/The-Spiral-Language You can also do a lot of this with constexpr / templates in C++, although the syntax is horrible. As you allude to, these staged programming systems can have TWO languages or one. In the case of Scala and Zig it's one language, but in the case of Lua/Terra it's two. Arguably in the case of C++, it's two languages and not one. Another thing to search for is "partial evaluation". I still have to understand the difference more precisely, as alluded to in the link above. https://en.wikipedia.org/wiki/Multi-stage_programming The Python/RPython language pair in PyPy is also related: https://morepypy.blogspot.com/2018/09/the-first-15-years-of-...

g does not depend on a particular type system or whether the language is dynamic or static. There are only so many good ideas in programming languages and this is one of them. Though there is some overlap between pattern matching and multiple dispatch, the roles are different. The purpose of multiple dispatch is extensibility, but the purpose of pattern matching is destructuring. https://stackoverflow.com/questions/2502354/what-is-pattern-... Note the great disparity in the F# examples that do pattern matching and the C# examples that do manual reflection. https://github.com/mrakgr/The-Spiral-Language/blob/0741959ac... Here are a few very simple examples of it in action in Spiral. I use them as compiler tests. https://github.com/mrakgr/The-Spiral-Language/blob/0741959ac... Here is quite a complex example of how it is used in action. I won't go into detail of this here, but you can see how I repeatedly match on the contents of a module at different times in order to get more generic functionality for the kernel. The particular kernel shown here is the most complex one that exists in the library right now - I am yet to get to things like generic matrix multiplication and convolution, but I'll get there eventually. > I see how it sounds nice, but I can't really imagine right now how it would improve my life. Let me just say that it is really difficult to know ahead of time how a particular language feature would affect your programming life. I could have said the same thing about first class functions back in 2015. I am sure in the future there will be such features I can't even imagine right now. > And if we don't get it as part of the compiler, we can definitely do things like that with https://github.com/jrevels/Cassette.jl/ I'll have to watch the talk on this. Thanks for the link.

I am doing currently which might take a while. https://github.com/mrakgr/The-Spiral-Language/blob/7ecd30bdf... You can see how it is implement here for the forward and backward parts. https://github.com/mrakgr/The-Spiral-Language/blob/7ecd30bdf... In the actual cost function, it is a bit different since I fuse the forward and the backward parts. It is a bit ugly because of all the type checking for whether the argument is a dual, but that is all done at compile time. > https://mikeinnes.github.io/2017/08/24/cudanative.html This is actually loop unrolling example that I had in mind when I said that Julia needs macros for. https://github.com/mrakgr/The-Spiral-Language#3-loops-and-ar... You can in fact get loop unrolling with just standard functions in Spiral. I go into it in the context of this chapter. It is achieved by pattern matching over tuples and recursion. > If you can make concrete examples of how things can be simpler than this, I'd be delighted to hear them :) Yes, by replacing meta programming with intensional polymorphism and inlinining guarantees. Also reflection should be done using pattern matching. I think this last one could be taken entirely seriously as it would not involve a replacement of the entire type system. All the claims in the article about inlining and specialization that make it sound like magic is what in general makes me dubious about languages pretending to be speed kings. Yes, I am aware that GPU kernels do not require optimizers capable of having monads for breakfast and that in the context of GPU programming where they were made they are probably true, but inlining is the sort of thing that matters more the more high level a language is. For very high level languages that desire speed, there isn't much choice but to make them a part of language semantics despite the added burden it puts on the user. Since we are still at it, I have a question I need to ask. Recently I've been informed that Julia is capable of GCing GPU memory. If this is fully integrated that would be a major feature which is not possible in say .NET or Racket. I really wanted this in Spiral and could not get it in .NET. By fully integrated, I mean much like for regular memory for which the GC takes note of the state of the system for when to do collection and defragmentation. If it can only make a thin wrapper with a finalizer (much like in .NET) around an unmanaged resource then it is not a big deal. Is it fully integrated or is it a wrapper style memory management? If it is the later, then that is too bad, but I'd suggest to Julia devs that they work on making it fully integrated as it would be a really good feature. Obviously, I can't do it in Spiral as I would need to write my own VM and I have only so many years in my life.

nd broadcasts in registers. It is also used to implement softmax for example. https://github.com/mrakgr/The-Spiral-Language/blob/3a94ca644...