Adventure Game Studio (AGS) module for lip sync
TotalLipSync is a script module for Adventure Game Studio (AGS). It provides voice-based lip sync by enabling games to play back speech animations that have been synchronized with voice clips.
TotalLipSync serves as an alternative to and replacement for the built-in AGS support for speech-based lip sync. The main advantages of TotalLipSync are:
TotalLipSync is tested with AGS v3.4.1 to 3.6.1. Compatibility with versions outside this range is unknown.
TLS_DUMMY
and give it exactly one loop and one frame.(* In versions of AGS before 3.6.0, instead of steps 3. and 4. you must place the lip sync data files in a folder called “sync/“ in the compiled game directory, e.g. “Compiled/Win/“.)
Initialize TotalLipSync:
TotalLipSync.Init(eLipSyncRhubarb);
TotalLipSync.AutoMapPhonemes();
Say a lip-synced line:
cSnarky.SaySync("&1 This line will play a lip sync animation.");
As with Character.Say()
, the “&1” prefix tells AGS to play this character, cSnarky
‘s, first speech clip. The filename of this clip should be snar1.wav
(or .mp3 or .ogg), taking the first four letters of the character name (without the leading c), followed by the line number (with no leading zeroes). Character.SaySync()
will also look for a corresponding lip sync file, snar1.pam
(or .dat, .anno, .tsv, or some other extension - depending on configuration) to read the lip sync data from. If there is no “&N” speech clip prefix or no corresponding lip sync data file can be found, no animation will play.
Further documentation and support (including example animations and explanations of lip syncing) is available on the AGS Forums.
void Character.SaySync(String message)
Works like Character.Say()
, but will also play a lip sync animation while the character speaks. The line must start with a “&N” speech clip prefix, and there must be a corresponding lip sync data file in the lip sync data directory.
void Character.SayAtSync(int x, int y, int width, String message)
Works like Character.SayAt()
, but will also play a lip sync animation while the character speaks. The line must start with a “&N” speech clip prefix, and there must be a corresponding lip sync data file in the lip sync data directory.
void TotalLipSync.Init(TotalLipSyncFileFormat lipSyncFormat, optional String fileExtension)
Initializes TotalLipSync and selects a file format for the lip sync data to parse. This method should be called before any others. You may call it again to change to a different format. The lip sync data file format options supported (and the file extensions used by default) are:
eLipSyncPamelaStressed
.pam
files)eLipSyncPamelaIgnoreStress
.pam
files)eLipSyncMoho
.dat
files)eLipSyncAnno
.anno
files)eLipSyncRhubarb
.tsv
files)The optional fileExtension
parameter lets you override the expected file extension (without changing what format the file data needs to be in).
void TotalLipSync.SetDataDirectory(String dataDirectory)
Sets the directory where TotalLipSync will look for lip sync data files. By defaults, this is “$DATA$/sync”, which means a directory “sync” in the project folder that is packaged within the game data. (When running in AGS versions below 3.6.0, the default is instead “$INSTALLDIR$/sync”.)
void TotalLipSync.SetFileExtension(String fileExtension)
Sets the extension of the lip sync data files to be read by TotalLipSync. The default is one of .pam
, .dat
, .anno
or .tsv
, depending on the data format chosen in TotalLipSync.Init()
.
void TotalLipSync.SetFileCasing(Casing fileCasing)
Sets the casing (capitalization) to use when looking for lip sync data filenames. This doesn’t make a difference if the system is case-insensitive (e.g. Windows) or if the directory is packaged within the game data, but does make a difference if the lip sync data files are read from the file system on a case-sensitive platform (e.g. Linux, MacOS).
The options are (using as an example the file for line &1 by character cRoger
, with data format set to Moho “.dat” files):
eCasingLowerCase
roge1.dat
)eCasingDefaultCase
Roge1.dat
)eCasingUpperCase
ROGE1.dat
)The casing does not apply to the file extension, which should instead be set using, e.g.:
TotalLipSync.SetFileExtension(".DAT");
void TotalLipSync.SetDataFileFrameRate(int frameRate)
Sets the frame rate of the lip sync data file. This value is used by the Pamela (.pam) and Moho Switch (.dat) formats. Default 24.
void TotalLipSync.SetSierraDummyView(int viewNumber)
Sets a dummy view that is used to enable lip sync for Sierra-style and full-screen speech modes. This view must have exactly 1 loop and 1 frame, and should not be used for anything else (since it will be overwritten by this module).
This is a workaround necessary because of the way Sierra-style speech works in AGS.
void TotalLipSync.SetDefaultFrame(int frameNumber)
Sets the speech view frame number to display when undefined in the sync file. (Typically corresponds to silence.) Should normally be 0.
void TotalLipSync.AutoMapPhonemes()
Sets up a default ten-frame mapping of phoneme codes to animation frames. The mapping depends on the data format selected in TotalLipSync.Init()
(because the different formats use different phoneme codes), but all map to a ten-frame SpeechView with the same ten frames.
See further documentation here.
void TotalLipSync.AddPhonemeMapping(String phoneme, int frame)
Adds a mapping from a phoneme code to an animation frame that will be displayed for this phoneme. Phoneme codes are case-insensitive.
void TotalLipSync.AddPhonemeMappings(String phonemes, int frame)
Adds mappings from a set of phoneme codes to an animation frame that will be displayed for those phonemes, separated by a slash ‘/‘. Phonemes are case-insensitive.
void TotalLipSync.ClearPhonemeMappings()
Clear all phoneme mappings.
Character* TotalLipSync.GetCurrentLipSyncingCharacter()
Returns the character that is currently being lip synced, or null
if none.
String TotalLipSync.GetCurrentPhoneme()
Returns the phoneme code that is currently active (i.e. the phoneme being spoken at this time). Returns null
if no character is being lip synced, and an empty String “” if a character is being lip synced but no phoneme has been set yet.
int TotalLipSync.GetCurrentFrame()
Returns the animation frame (i.e. the mouth shape) that is currently being displayed. -1 if no character is currently being lip synced.
v0.6
v0.5
v0.4