The Universal Shogi Interface, draft 1 (2007-01-24)

1. Introduction

This is a preliminary draft of the Universal Shogi Interface, a protocol for communication between a shogi engine and GUI. It is my belief that such a protocol, if widely adopted, would give several benefits for the computer shogi community:

The USI protocol, as well as the textual description on the protocol below, is based on the UCI protocol used in computer chess. Almost all the current top computer chess programs support the UCI protocol, and compatible GUIs exist for Windows, Linux and Mac OS X.

I am grateful to Stefan Meyer Kahlen, the author of the UCI protocol, for allowing me to use his work as the basis for my shogi protocol. Most of the text in section 5 below is copied verbatim from the official UCI protocol description.

2. Some notes about language

All communication between the engine and the GUI is done by 7-bit ASCII text, and the English language is used in the names of the various commands. English letters are used to denote the various piece types ("P" for pawn, "L" for lance, etc.). This may seem like a strange choice (at least to non-technical users), considering that the vast majority of users of shogi software are Japanese.

The justification for the use of 7-bit ASCII is to make the work easier for the engine programmer. Most shogi engines are written in low-level programming languages like C, where reading and writing Unicode is difficult, especially while trying to maintain portability. Translation of the move output from the engine into Japanese notation is assumed to be the responsibility of the GUI.

A USI engine may also have several engine parameters which can be configured by the user (e.g. style of play, piece values for the various pieces, different kinds of search parameters, and so on). Each parameter has an internal name in 7-bit ASCII which is used in all communication between the engine and the GUI. An USI engine should, in addition to the executable file, be delivered with a Unicode text file containing Japanese translations (and optionally translations into other languages) of all option names. The GUI should use the translations in this file when displaying the options to the user.

The format of the translations file is not yet decided, and will be described in a future version of this document.

3. Forsyth-Edwards notation for shogi

In computer chess, there is a standardized compact ASCII notation for chess positions, known as the Forsyth-Edwards notation (usually abbreviated as FEN). It was originally invented by David Forsyth more than 100 years ago, and was extended for use with chess software by Steven Edwards some time in the 1990s. For a precise definition of the FEN standard, consult section 16.1 of the PGN specification.

As far as I know, no equivalent standard exists for shogi (please correct me if I am wrong). This section describes a variant of FEN notation suitable for shogi.

A Shogi FEN string (SFEN from now on) consists of a single line of ASCII text containing four data fields, separated by whitespace. These are:

  1. Board state. The placement of the pieces of the board is written rank by rank, with each rank separated by a slash ('/'). The ranks are ordered from the white side, beginning with rank 'a' and ending with rank 'i' (using the standard English shogi board coordinates). For each rank, the squares are specified from file 9 to file 1. The uppercase letters "PLNSGBRK" are used for the non-promoted black pieces, while the lowercase letters "plnsgbrk" are used for the non-promoted white pieces. A promoted piece is denoted by a '+' character immediately before the piece letter, e.g. +B for a black promoted bishop. One or more contiguous empty squares are represented by a digit counting the number of squares.
  2. The side to move. This field contains of a single letter, "b" (for "black") or "w" (for "white"), depending on the current side to move.
  3. Pieces in hand.. Again we use uppercase letters for black pieces, and lowercase letters for white's captured pieces. A digit before a letter means that there are several pieces of the given type in hand. The pieces are always listed in the order rook, bishop, gold, silver, knight, lance, pawn; and with all black pieces before all white pieces. As an example, in a position where black has one rook, one gold and four pawns in hand, while white has two bishops, two silvers and three pawns, the pieces in hand data field in the SFEN would look like "RG4P2b2s3p". If neither player has any pieces in hand, a single minus character ("-") is used for the pieces in hand field.
  4. Move count. An integer representing the number of the current move in the game. We are using the shogi convention for move counting, which means that we count what international players would call "plies" or "half moves". For instance, after the moves 1. P7g-7f 2. P8c-8d, the move counter is 3. The "move count" data field is optional; a program should be able to read and understand an SFEN string even if this field is missing.

As an example, the initial position in shogi is encoded like this:

lnsgkgsnl/1r5b1/ppppppppp/9/9/9/PPPPPPPPP/1B5R1/LNSGKGSNL b - 1

A more complicated example is the following position, taken from the 3rd game of the 19th Ryu-O match between Sato and Watanabe:


  9    8    7    6    5    4    3    2    1
+----+----+----+----+----+----+----+----+----+
|    |    |    |    |    |    |    |    | Lw | a    1B 1G 1N 3P
+----+----+----+----+----+----+----+----+----+
|    | Lw |+Rb |    |    | Pb |    |    |    | b
+----+----+----+----+----+----+----+----+----+
| Pw |    |    | Pw | Bb | Gb |    | Pw | Pw | c
+----+----+----+----+----+----+----+----+----+
| Kw | Pw | Sw |    | Pw |    |    |    |    | d
+----+----+----+----+----+----+----+----+----+
| Nb | Nw |    | Pb |    |    | Gb |    |    | e
+----+----+----+----+----+----+----+----+----+
| Pb |    | Pb |    | Pb |    |    | Pb | Pb | f
+----+----+----+----+----+----+----+----+----+
|    | Pb | Sb |    |    |    |    |    |    | g
+----+----+----+----+----+----+----+----+----+
|    | Kb | Sb | Gb |    |    |    |+Rw |    | h
+----+----+----+----+----+----+----+----+----+
| Lb | Nb |    |    |+Pw |    |    |    | Lb | i    1S 
+----+----+----+----+----+----+----+----+----+

The SFEN for the position above is:

8l/1l+R2P3/p2pBG1pp/kps1p4/Nn1P2G2/P1P1P2PP/1PS6/1KSG3+r1/LN2+p3L w Sbgn3p 124

The SFEN notation described above differs from the FEN notation for chess positions in a few ways. The castling and en passant fields have been removed, for the obvious reason that castling and en passant captures do not exist in shogi. The halfmove clock has been removed because shogi has no 50-move draw rule. Finally, because captured pieces can be dropped back on the board in shogi, the "pieces in hand" data field has been added.

4. Differences between the UCI and USI protocols

This section is intended for programmers who are already familiar with the UCI protocol. Those who do not know anything about UCI are adviced to skip to section 5.

The USI protocol is identical to the USI protocol apart from the following changes:

5. Description of the universal shogi interface

5.1. General rules

5.2. Move format

5.3. GUI to engine

These are all the command the engine gets from the interface.

usi

Tell engine to use the USI (universal shogi interface). This will be sent once as a first command after program boot to tell the engine to switch to USI mode. After receiving the usi command the engine must identify itself with the id command and send the option commands to tell the GUI which engine settings the engine supports. After that, the engine should send usiok to acknowledge the USI mode. If no usiok is sent within a certain time period, the engine task will be killed by the GUI.

debug [ on | off ]

Switch the debug mode of the engine on and off. In debug mode the engine should send additional infos to the GUI, e.g. with the info string command, to help debugging, e.g. the commands that the engine has received etc. This mode should be switched off by default and this command can be sent any time, also when the engine is thinking.

isready

This is used to synchronize the engine with the GUI. When the GUI has sent a command or multiple commands that can take some time to complete, this command can be used to wait for the engine to be ready again or to ping the engine to find out if it is still alive. This command is also required once before the engine is asked to do any search to wait for the engine to finish initializing. This command must always be answered with readyok and can be sent also when the engine is calculating in which case the engine should also immediately answer with readyok without stopping the search.

setoption name <id> [value <x>]

This is sent to the engine when the user wants to change the internal parameters of the engine. For the button type no value is needed. One string will be sent for each parameter and this will only be sent when the engine is waiting. The name and value of the option in <id> should not be case sensitive and can not include spaces.

register

This is the command to try to register an engine or to tell the engine that registration will be done later. This command should always be sent if the engine has sent registration error at program startup.

The following tokens are allowed:

later
The user doesn't want to register the engine now.
name <x>
The engine should be registered with the name <x>
code <y>
The engine should be registered with the code <y>

Example:


	"register later"
	"register name Stefan MK code 4359874324"
    
usinewgame

This is sent to the engine when the next search (started with position and go) will be from a different game. This can be a new game the engine should play or a new game it should analyse but also the next position from a testsuite with positions only. As the engine's reaction to usinewgame can take some time the GUI should always send isready after usinewgame to wait for the engine to finish its operation.

position [sfen <sfenstring> | startpos ] moves <move1> ... <movei>

Set up the position described in sfenstring on the internal board and play the moves on the internal board. If the game was played from the start position, the string startpos will be sent.

Note: If this position is from a different game than the last position sent to the engine, the GUI should have sent a usinewgame inbetween.

go

Start calculating on the current position set up with the position command. There are a number of commands that can follow this command, all will be sent in the same string.

If one command is not sent its value should be interpreted as if it would not influence the search.

searchmoves <move1> ... <movei>

Restrict search to this moves only

Example: After position startpos and go infinite searchmoves 7g7f 2g2f, the engine should only search the two moves P-7f and P-2f in the initial position.

ponder

Start searching in pondering mode. Do not exit the search in ponder mode, even if it's mate! This means that the last move sent in in the position string is the ponder move. The engine can do what it wants to do, but after a ponderhit command it should execute the suggested move to ponder on. This means that the ponder move sent by the GUI can be interpreted as a recommendation about which move to ponder. However, if the engine decides to ponder on a different move, it should not display any mainlines as they are likely to be misinterpreted by the GUI because the GUI expects the engine to ponder on the suggested move.

btime <x>
Black has x milliseconds left on the clock
wtime <x>
White has x milliseconds left on the clock
binc <x>
Black increment per move i milliseconds if x > 0.
winc <x>
White increment per move i milliseconds if x > 0.
movestogo <x>
There are x moves to the next time control. This will only be sent if x > 0. If you don't get this and get the wtime and btime, it's sudden death.
depth <x>
Search x plies only.
nodes <x>
Search x nodes only.
mate <x>
Search for a mate in x moves. Programmers coming from computer chess should note that we follow the shogi convention for counting moves, i.e. we count what chess players usually describe as "plies" or "half moves". For instance, mate 5 means mate in 5 plies.
movetime <x>
Search exactly x milliseconds.
infinite
Search until the stop command is received. Do not exit the search without being told so in this mode!
stop

Stop calculating as soon as possible. Don't forget the bestmove and possibly the ponder token when finishing the search.

ponderhit

The user has played the expected move. This will be sent if the engine was told to ponder on the same move the user has played. The engine should continue searching but switch from pondering to normal search.

quit

Quit the program as soon as possible.

5.3. Engine to GUI

id
name <x>
This must be sent after receiving the usi command to identify the engine, e.g. id name Shredder X.Y\n
author <x>
This must be sent after receiving the usi command to identify the engine, e.g. id author Stefan MK\n
usiok

Must be sent after the id and optional options to tell the GUI that the engine has sent all infos and is ready in usi mode.

readyok

This must be sent when the engine has received an isready command and has processed all input and is ready to accept new commands now. It is usually sent after a command that can take some time to be able to wait for the engine, but it can be used anytime, even when the engine is searching, and must always be answered with readyok.

bestmove <move1> [ponder <move2>]

The engine has stopped searching and found the move <move> best in this position. The engine can send the move it likes to ponder on. The engine must not start pondering automatically. this command must always be sent if the engine stops searching, also in pondering mode if there is a stop command, so for every go command a bestmove command is needed! Directly before that the engine should send a final info command with the final search information, the the GUI has the complete statistics about the last search.

copyprotection

This is needed for copyprotected engines. After the usiok command the engine can tell the GUI, that it will check the copy protection now. This is done by copyprotection checking. If the check is ok the engine should send copyprotection ok, otherwise copyprotection error.

If there is an error the engine should not function properly but should not quit alone. If the engine reports copyprotection error the GUI should not use this engine and display an error message instead!

The code in the engine can look like this:


	TellGUI("copyprotection checking\n");
	// ... check the copy protection here ...
	if(ok)
	  TellGUI("copyprotection ok\n");
	else
          TellGUI("copyprotection error\n");
    
registration

This is needed for engines that need a username and/or a code to function with all features. Analogously to the copyprotection command the engine can send registration checking after the usiok command followed by either registration ok or registration error. Also after every attempt to register the engine it should answer with registration checking and then either registration ok or registration error.

In contrast to the copyprotection command, the GUI can use the engine after the engine has reported an error, but should inform the user that the engine is not properly registered and might not use all its features.

In addition the GUI should offer to open a dialog to enable registration of the engine. To try to register an engine the GUI can send the register command. The GUI has to always answer with the register command if the engine sends registration error at engine startup (this can also be done with register later) and tell the user somehow that the engine is not registered. This way the engine knows that the GUI can deal with the registration procedure and the user will be informed that the engine is not properly registered.

info

The engine wants to send information to the GUI. This should be done whenever one of the info has changed.

The engine can send only selected infos or multiple infos with one info command, e.g. info currmove 2g2f currmovenumber 1 or info depth 12 nodes 123456 nps 100000.

Also all infos belonging to the pv should be sent together, e.g.


	info depth 2 score cp 214 time 1242 nodes 2124 nps 34928 pv 2g2f 8c8d 2f2e
    

I suggest to start sending currmove, currmovenumber, currline and refutation only after one second in order to avoid too much traffic.

Additional info:

depth <x>
Search depth in plies.
seldepth <x>
Selective search depth in plies. If the engine sends seldepth there must also be a depth present in the same string.
time <x>
The time searched in ms. This should be sent together with the pv.
nodes <x>
x nodes searched. The engine should send this info regularly.
pv <move1> ... <movei>
The best line found.
multipv <num>
This for the multi pv mode. For the best move/pv add multipv 1 in the string when you send the pv. In k-best mode, always send all k variants in k strings together.
score
cp <x>
The score from the engine's point of view, in centipawns.
mate <y>
Mate in y plies. If the engine is getting mated, use negative values for y.
lowerbound
The score is just a lower bound.
upperbound
The score is just an upper bound.
currmove <move>
Currently searching this move.
currmovenumber <x>
Currently searching move number x, for the first move x should be 1, not 0.
hashfull <x>
The hash is x permill full. The engine should send this info regularly.
nps <x>
x nodes per second searched. the engine should send this info regularly.
cpuload <x>
The cpu usage of the engine is x permill.
string <str>
Any string str which will be displayed be the engine. if there is a string command the rest of the line will be interpreted as <str>.
refutation <move1> <move2> ... <movei>
Move <move1> is refuted by the line <move2> ... <movei>, where i can be any number >= 1. Example: after move 8h2b+ is searched, the engine can send info refutation 8h2b+ 1c2b if 1c2b is the best answer after 8h2b+ or if 1c2b refutes the move 8h2b+. If there is no refutation for 8h2b+ found, the engine should just send info refutation 8h2b+. The engine should only send this if the option USI_ShowRefutations is set to true.
currline <cpunr> <move1> ... <movei>
This is the current line the engine is calculating. <cpunr> is the number of the cpu if the engine is running on more than one cpu. <cpunr> = 1,2,3.... If the engine is just using one cpu, <cpunr> can be omitted. If <cpunr> is greater than 1, always send all k lines in k strings together. The engine should only send this if the option USI_ShowCurrLine is set to true.
option

This command tells the GUI which parameters can be changed in the engine. This should be sent once at engine startup after the usi and the id commands if any parameter can be changed in the engine. The GUI should parse this and build a dialog for the user to change the settings. Note that not every option should appear in this dialog, as some options like USI_Ponder, USI_AnalyseMode, etc. are better handled elsewhere or are set automatically.

If the user wants to change some settings, the GUI will send a setoption command to the engine.

Note that the GUI need not send the setoption command when starting the engine for every option if it doesn't want to change the default value. For all allowed combinations see the examples below, as some combinations of this tokens don't make sense.

One string will be sent for each parameter.

name <id>

The option has the name <id>. Whitespace is not allowed in an option name. Note that the name should normally not be displayed directly in the GUI: The GUI should look up the option name in the translation file, and present the translation into the users preferred language in the engine's option dialog.

Certain options have a fixed value for <id>, which means that the semantics of this option is fixed. Usually those options should not be displayed in the normal engine options window of the GUI but get a special treatment. USI_Pondering for example should be set automatically when pondering is enabled or disabled in the GUI options. The same for USI_AnalyseMode which should also be set automatically by the GUI. All those certain options have the prefix USI_. If the GUI gets an unknown option with the prefix USI_, it should just ignore it and not display it in the engine's options dialog.

The options with fixed semantics are:

<id> = USI_Hash, type spin
The value in MB for memory for hash tables can be changed, this should be answered with the first setoptions command at program boot if the engine has sent the appropriate option name Hash command, which should be supported by all engines! So the engine should use a very small hash first as default.
<id> = USI_Ponder, type check
This means that the engine is able to ponder (i.e. think during the opponent's time). The GUI will send this whenever pondering is possible or not. Note: The engine should not start pondering on its own if this is enabled, this option is only needed because the engine might change its time management algorithm when pondering is allowed.
<id> = USI_OwnBook, type check
This means that the engine has its own opening book which is accessed by the engine itself. If this is set, the engine takes care of the opening book and the GUI will never execute a move out of its book for the engine. If this is set to false by the GUI, the engine should not access its own book.
<id> = USI_MultiPV, type spin
The engine supports multi best line or k-best mode. The default value is 1.
<id> = USI_ShowCurrLine, type check
The engine can show the current line it is calculating. See info currline above. This option should be false by default.
<id> = USI_ShowRefutations, type check
The engine can show a move and its refutation in a line. See info refutations above. This option should be false by default.
<id> = USI_LimitStrength, type check
The engine is able to limit its strength to a specific dan/kyu number. This should always be implemented together with USI_Strength. This option should be false by default.
<id> = USI_Strength, type spin
The engine can limit its strength within the given interval. Negative numbers are kyu levels, while positive numbers are amateur dan levels. If USI_LimitStrength is set to false, this value should be ignored. If USI_LimitStrength is set to true, the engine should play with this specific strength. This option should always be implemented together with USI_LimitStrength.
<id> = USI_AnalyseMode, type check
The engine wants to behave differently when analysing or playing a game. For example when playing it can use some kind of learning, or an asymetric evaluation function. The GUI should set this option to false if the engine is playing a game, and to true if the engine is analysing.
type <t>

The option has type t. There are 5 different types of options the engine can send:

check
A checkbox that can either be true or false.
spin
A spin wheel or slider that can be an integer in a certain range.
combo
A combo box that can have different predefined strings as a value.
button
A button that can be pressed to send a command to the engine
string
A text field that has a string as a value, an empty string has the value <empty>.
filename
Similar to string, but is presented as a file browser instead of a text field in the GUI.
default <x>

The default value of this parameter is x.

min <x>

The minimum value of this parameter is x.

max <x>

The maximum value of this parameter is x.

Here are some examples illustrating the different types of options:


	"option name Nullmove type check default true\n"
	"option name Selectivity type spin default 2 min 0 max 4\n"
	"option name Style type combo default Normal var Solid var Normal var Risky\n"
	"option name LearningFile type filename default /shogi/my-shogi-engine/learn.bin"
	"option name ResetLearning type button\n"
    

5.4. Examples

This is how the communication when the engine boots can look like:



GUI     engine
// tell the engine to switch to USI mode
usi

// engine identify  

        id name Glaurung Shogi 0.1
        id author Tord Romstad

// engine sends the options it can change
// the engine can change the hash size from 8 to 1024 MB

        option name USI_Hash type spin default 16 min 8 max 1024

// the engine can switch off Nullmove and set the playing style

        option name Nullmove type check default true
        option name Style type combo default Normal var Solid var Normal var Risky
// the engine has sent all parameters and is ready

        usiok

// Note: here the GUI can already send a "quit" command if it just
// wants to find out details about the engine, so the engine should
// not initialize its internal parameters before here.

// now the GUI sets some values in the engine

// set hash to 128 MB

setoption name USI_Hash value 128

// set play style to "Solid"

setoption name Style value Solid


// waiting for the engine to finish initializing

// this command and the answer is required here!

isready

// engine has finished setting up the internal values

        readyok

// now we are ready to go

// if the GUI is supporting it, tell the engine that is is
// searching on a game that it hasn't searched on before

usinewgame

// if the engine supports the "USI_AnalyseMode" option and the next
// search is supposed to be an analysis, the GUI should set
// "USI_AnalyseMode" to true if it is currently set to false with this 
// engine.

setoption name USI_AnalyseMode value true

// tell the engine to search infinite from the start position after
// 1. P-7f 2. P-3d:

position startpos moves 7g7f 3c3d
go infinite

// the engine starts sending infos about the search to the GUI

// (only some examples are given)

        info depth 1
        info score cp -2 depth 1 nodes 13 time 5 pv 8h2b+ 3a2b
        info depth 1
        info score cp 13 depth 1 nodes 24 time 11 pv 2g2f
	info depth 2
        info nps 15937
        info score cp 5 depth 2 nodes 255 time 90 pv 2g2f 4c4d
        info depth 3 seldepth 7
        info nps 26437
        info score cp 20  depth 3 nodes 1123 time 315 pv 2g2f 4c4d 2f2e
        ...

// here the user has seen enough and asks to stop the searching

stop

// the engine has finished searching and is sending the bestmove command
// which is needed for every "go" command sent to tell the GUI
// that the engine is ready again

        bestmove 2g2f ponder 4c4d