Questions and answers about Pepper3

Series: Examples, Questions

My last post Examples of Pepper3 code was a reply to my friend’s email asking what it was all about. They replied with some questions, and I thought the questions and answers might shed some more light:

Questions!

Brilliant ones, thanks.

In general though you’ve said a lot about what Pepper can do without giving design decisions.

Yep, total brain dump.

Remind me again who this language is for :)

It’s a multi-paradigm (generic, functional, OO) language aimed at application programmers who want:

  • “native” performance on their chosen platform (definitely including actual native machine code). This is inspired by C++.
  • easy deployment (preferably a single binary containing everything, with an option to link most dependencies statically), including packaging of installers for major OSes. This is inspired by C++, and the pain of C++.
  • perfect flexibility for creating types – “meta-programming” is just programming. Things you would have done using code generation (e.g. generating a class hierarchy from an XSD) are done by running arbitrary code at compile time. The powerful type system is inspired by Haskell and the book “Modern C++ Design”, and the meta-programming is inspired by Lisp.
  • Simple memory management without GC through ownership. This is inspired by modern C++, and then Rust came along and implemented it before I could, thus proving it works. However, I would remove a lot of the functionality in Rust (lifetimes) to make it much simpler.
  • Strong support for functional programming if you want it. This is inspired by Haskell.
  • The simplest possible core language, with application programmers able to expand it by giving them the same tools as the language designers – e.g. “for” is just a function, so you can make your own. I am hoping I can even make “class” a function. This is inspired by Lisp, and oppositely-inspired by Java.
  • Separation between the idea of Interfaces, which I think I will call “type specifiers” (and will allow arbitrary code execution to determine whether a type satisfies the requirements) and structs/classes, allowing us to make new Interfaces and have old code satisfy them, meaning we can do generic stuff with e.g. ints even if
    no-one declared that “class Int : public Quaternion” or whatever.
  • Lots of “nudges” towards things that are good: by default things will be functional and immutable – you will have to explicitly say if you want to use more dangerous constructs like side effects and mutable values.
  • No implicit conversions, or really anything happening without you saying so.

Can you assign floats to ints or vice versa?

Yes, but you shouldn’t.

If you’re setting types in code at the start of a file, is this only available in the main file? Are there multiple files per program? Can
you have libraries? If so, do these decide the functionality of their types in the library or does this only happen in the main file?

I haven’t totally decided – either by being enforced, or as a matter of style, you will generally do this once at the beginning of the program (and choose on the compiler command line to do it e.g. the debug way or the release way) and it will affect all of your code.

Libraries will be packaged as Pepper3 source code, so choices you make of the type of Int etc. will be reflected through the whole dependency tree. Cool, huh?

This is inspired by Python.

Can you group variables together into structs or similar?

Yes – it will be especially easy to make “value types”, and lots of default methods will be provided, that you will be strongly encouraged to use – e.g. copy and move operations. This is inspired by Elm.

Why are variables immutable by default but mutable with a special syntax? It’s the opposite of C++ const, but why that way around?

This is one of the “nudges” – immutable stuff is much easier to think about, and makes parallel stuff easier, and allows optimisations and so on, so turning it on by default means you have to choose to take the bad path, and are inclined to take the virtuous one. This is inspired by Haskell and Rust.

Why only allow assignments, function calls and operators? I’m sure you have good reasons.

To be as simple as possible, so you only have those things to learn and the rest can be understood by just reading the code. This is inspired by Python.

I wrote more of my (earlier) thoughts in this 4-post series, which is better thought through: Goodness in Programming Languages

Examples of Pepper3 code

Series: Examples, Questions

I have restarted my effort to make a new programming language that fits the way I like things. I haven’t pushed any code yet, but I have made a lot of progress in my head to understand what I want.

Here are some random examples that might get across some of the ways I am thinking:


// You code using general types like "Int" but you can set what
// they really are in the code (usually at the beginning), so
// if you plan to use native ints in the production code, it's
// a good idea to use:
Int = CheckedNativeInt;
// while in dev, since it will crash at runtime if you overflow.

// Then, in production when you're sure you have no errors,
// switch to an unchecked one:
Int = NativeInt;

// But, if you prefer correctness over efficiency, you can use
// mathematical integer that never overflows:
Int = ArbitrarySizeInt;


// Variables are immutable by default, so:
Int x = 4;
x = 3;      // this is a compile error


// But this is OK
Mutable(Int) y = 6;
y = y + x;

// Notice that you can call functions that return types that you
// then use, like Mutable(Int) here.

// Generally, code can run at either compile time or run time.
// Code to do with types has to run at compile time.
// By default, other code runs at run time, but you can force
// it to run early if you want to.


// A main method looks like this - you get hold of e.g. stdout through
// a World instance - I try to avoid any global functions like print, or
// global variables like sys.stdout.

Auto main =
{:(World world)->Int
    //...
};

// (Although note that Int, String etc. actually are global variables,
// which is a bit annoying)

// I wish the main method were simpler-looking.  The only saving grace
// is that for simple examples you don't need a main method -
// Pepper3 just calculates the expression you provide in your file and
// prints it out.


// Expressions in curly brackets are lambda functions, so:

{3};

// is a function taking no arguments, returning 3, and:

{:(Int x)
    x * 2
};

// is a function that doubles a value.

Obviously, we can tie functions to names:

Auto dbl =
    {:(Int x)
        x * 2
    };

// Meaning we can call dbl like this:
dbl(4);

// Auto is a magic word to say ("use type inference"), so
// this is equivalent to the above:

fn([Int]->Int) dbl =
    {:(Int x)
        x * 2
    };


// Because {} makes an anon function, things like "for" can be
// functions instead of keywords.

for(range(3), {:(Int x)
    world.stdout.println(to(String)(x));
});


// As far as possible, Pepper3 will only contain assignment statements:
String s = "xx";

// and expressions containing function calls and operators:
dbl(3) + 6;


// This means we can make our own constructs like a different type of
// for loop, which would need a new keyword in some languages:

Auto parallel_for = import(multiprocess.parallel_for);

Recording gameplay videos on RetroPie

Credits: this is a slightly corrected and shortened version of How To Record A GamePlay Video From A RetroPie by selsine, which is itself based on Recording Live Gameplay in RetroPie’s RetroArch Emulators Natively on the Raspberry Pi by Retro Resolution.

RetroPie is based on RetroArch. RetroArch has a feature to record gameplay videos, but the current version of RetroPie has it disabled, presumably because it was thought to be too intensive to run properly on a Raspberry Pi.

These instructions tell you how to turn the recording feature on, and set it up. This works perfectly on my Raspberry Pi 3, allowing me to record video and sound from games I am playing.

The code for this is here: github.com/andybalaam/retropie-recording – this was code written by RetroRevolution, with small corrections and additions by me.

Before you start, you should have RetroPie working and connected to the Internet, and updated to the latest version.

Note: you should make a backup of your RetroPie before you start, because if you type the command below you could completely break it, meaning you will have to wipe your SD card and start fresh.

Turning on the recording feature

RetroArch uses the ffmpeg program to record video. To turn on recording, we need to log into the Pi using ssh, download and compile ffmpeg, and then recompile RetroArch with recording support turned on.

Log in to the Pi using ssh

Find out the IP address of your Pi by choosing “RetroPie setup” in the RetroPie menu and choosing “Show IP Address”. Write down the IP address (four numbers with dots in between – for example: 192.168.0.3).

On your Linux* computer open a Terminal and type:

ssh pi@192.168.0.3

(put in the IP address you wrote down instead of 192.168.0.3)

When it asks for your password, type: raspberry

If this works right, you should see something like this:
The RetroPie Project joystick logo

* Note: if you don’t have Linux, this should work OK on a Mac, or on Windows you could try using PuTTY.

Download and compile ffmpeg

Log in to the RetroPie as described above. The commands shown below should all be typed in to the window where you are logged in to the RetroPie.

Download the script ffmpeg-install.sh by typing this:

wget https://github.com/andybalaam/retropie-recording/raw/master/ffmpeg-install.sh

Now run it like this:

bash ffmpeg-install.sh

(Note: DON’T use sudo to run this – just type exactly what is written above.)

Now wait a long time for this to work. If it prints out errors, something went wrong – read what it says, and you may need to edit the ffmpeg-install.sh script to figure out what to do. Leave a comment and include the errors you saw if you need help.

Hopefully it will end successfully and print:

FFmpeg and Codec Installation Complete

If so, you are ready to move on to recompiling RetroArch:

Recompile RetroArch with recording turned on

Download the script build-retroarch-with-ffmpeg.sh by typing this:

wget https://github.com/andybalaam/retropie-recording/raw/master/build-retroarch-with-ffmpeg.sh

Now run it like this:

bash build-retroarch-with-ffmpeg.sh

It should finish in about 10 minutes, and print:

Building RetroArch with ffmpeg enabled complete

If it printed that, your RetroPie now has recording support! Restart your RetroPie:

Restart the RetroPie

Restart your RetroPie.

If you want to check that recording support is enabled, Look for “Checking FFmpeg Has Been Enabled in RetroArch” on the RetroResolution guide.

Now you need to set up RetroPie to record your emulator.

Setting up recording for your emulator

To set up an emulator, you need a general recording config file (the same for all emulators), and a launch config for the actual emulator you are using.

Create the recording config file

Log into the RetroPie as described in the first section, and type this to download the recording config file. If you want to change settings like what file format to record in, this is the file you will need to change.

wget https://github.com/andybalaam/retropie-recording/blob/master/recording-config.cfg

Create a launch config for your emulator

Each RetroPie emulator has a config file that describes how to launch it. For example, the NES emulator’s version is in /opt/retropie/configs/nes/emulators.cfg.

To get a list of all the emulators, log into your RetroPie and type:

ls /opt/retropie/configs

In that list you will see, for example, “nes” for the NES emulators, and “gb” for the GameBoy emulators. Find the one you want to edit, and edit it with the nano editor by typing:

nano /opt/retropie/configs/gb/emulators.cfg

(Instead of “gb” type the right name for the emulator you want to use, from the list you got when you typed the “ls” command above.)

Now you need to add a new line in this file. Each line describes how to launch an emulator. You should copy an existing line, and add some more stuff to the end.

For example, my version of this file looks like this:

lr-gambatte = "/opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-gambatte/gambatte_libretro.so --config /opt/retropie/configs/gb/retroarch.cfg %ROM%"
lr-gambatte-record = "/opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-gambatte/gambatte_libretro.so --config /opt/retropie/configs/gb/retroarch.cfg --record /home/pi/recording_GB_$(date +%Y-%m-%d-%H%M%S).mkv --recordconfig /home/pi/recording-config.cfg %ROM%"
default = "lr-gambatte"
lr-tgbdual = "/opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-tgbdual/tgbdual_libretro.so --config /opt/retropie/configs/gb/retroarch.cfg %ROM%"

The line I added is coloured: The green parts are things copied from the line above, and the red parts are new – those parts tell the launcher to use the recording config we made in the previous section.

When you’ve made your edits, press Ctrl-X to exit nano, and type “Y” when it asks whether you want to save.

Once you’ve done something similar to this for every emulator you want to record with, you are ready to actually do the recording!

Actually doing a recording

Launching a game with recording turned on

In the normal RetroPie interface, go to your emulator and start it, but press the A button while it’s launching, and choose “Select emulator for ROM”. In the list that comes up, choose the new line you added in emulators.cfg. In our example, that was called “lr-gambatte-record”.

Now play the game, and exit when you are finished. If all goes well, the recording will have been saved!

(Note: doing this means that every time you launch this game it will be recorded. To stop it doing this, press the “A” button while it’s launching, choose “Select emulator for ROM” and choose the normal line – in our example that would be “lr-gambatte”.)

Getting the recorded files

To get your recording off the RetroPie, go back to your computer, open a terminal, and type:

scp pi@192.168.0.3:recording_*.mkv ./

This will copy all recorded videos from your RetroPie onto your computer (into your home directory, unless you did a cd commmand before you typed the above).

Now you should delete the files from your RetroPie. Log in to the RetroPie as described in the first section, and delete all recording files by typing this:

Note: This deletes all your recordings, and you can’t undo!

rm recording_*.mkv

Note: This deletes all your recordings, and you can’t undo!

Safer: recording onto a USB stick

Note: recording directly onto the RetroPie like we described above is dangerous because you could fill up all the disk space or corrupt your SD card, which could make RetroPie stop working, meaning you need to wipe your SD card and set up RetroPie again.

It’s safer to record onto a separate USB disk. To find out how, read “Recording to an External Storage Device” in Retro Resolution’s guide.

TECH(K)NOW Day workshop on “Writing a programming language”

My OpenMarket colleagues and I ran a workshop at TECH(K)NOW Day on how to write your own programming language:

A big thank you to my colleagues from OpenMarket who volunteered to help: Rowan, Jenny, Zach, James and Elliot.

An extra thank you to Zach and Elliott for their impromptu help on the information desk for attendees:

Hopefully the attendees enjoyed it and learned a bit:

You can find the workshop slides, the full code, info about another simple language called Cell, and lots more links here: github.com/andybalaam/videos-write-your-own-language, my blog at artificialworlds.net/blog, and follow me on twitter @andybalaam.

Thanks to OpenMarket for supporting us in running this workshop!