Expect for Windows® User Guide

Using ActiveState Expect for Windows

• Expect for Windows Overview
• • About this Document
• Writing Expect Scripts
• • Using Expect with Tcl
  • Use Case 1: FTP
  • Use Case 2: Updating Passwords
  • Use Case 3: Remote Directory Lister Tutorial
• Porting Expect Scripts to Windows
• • Expect for Windows Commands
  • Modified Commands and Behaviors
  • Unimplemented Commands and Behaviors
  • Caveats
  • Tips for Writing Cross-Platform Expect Scripts
• Wrapping Expect using TclApp

Top

Expect for Windows Overview

ActiveState Expect for Windows provides the ability to automate interactive programs from within the Windows environment. For example, use Expect for Windows to telnet to a Unix client and run commands on the remote system. By driving the telnet session with Expect for Windows, you have the ability to fully automate the process.

ActiveState Expect for Windows includes the following features:

• Automation of interactive programs from the Windows platform.
• Compatibility with the latest Tcl core (Tcl 8.4+).
• Stubs-enabled and deployable using TclApp, a component in the ActiveState Tcl Dev Kit.

About this Document

This document discusses how to use Expect for Windows to automate various programs. This document is divided into three parts:

• The first part describes three Expect for Windows use cases.
• The second part addresses how the Expect for Windows port differs from Expect, including command and behavior modifications. Tips for writing cross-platform scripts are also outlined.
• The third part describes the process of wrapping Expect for Windows scripts in TclApp, a component of the Tcl Dev Kit.

This document should be used together with the Expect for Windows Reference, which lists all available Expect commands and options.

Top

Writing Expect Scripts

This section discusses how to write Tcl scripts to use Expect for Windows. The first part addresses using Expect with Tcl. Next, three use cases are provided to demonstrate how Expect for Windows can be used to automate the interaction between various remote Unix programs. Each use case includes code snippets to demonstrate the communication between Expect for Windows and the interacting Unix program. It should be noted that this sample code is incomplete and does not provide the adequate error checking recommended for successful automation.

Using Expect with Tcl

Expect for Windows is implemented as a Tcl package (as opposed to a stand-alone executable). Expect for Windows must be declared in a Tcl script with the package require Expect statement. For example:

    #!/bin/sh
    # \
    exec tclsh "$0" ${1+"$@"}
    package require Expect

Failure to declare the package require Expect statement results in an error message when running the script on the Windows platform.

Use Case 1: FTP

Scenario:
Use Expect for Windows to automatically upload a completed software build from a Windows machine to a Unix FTP server.

      # 1. Set the ftp prompt using a regular expression.
      set ftp_prompt "ftp>?"

      # 2. Print to console.
      puts stdout "Connecting to $hostname to upload the latest build."

      # 3. Connect to the FTP server using the "spawn" command.
      spawn ftp $hostname

      # 4. Wait for a login prompt.
      expect -re "(Name|login|Login|Username).*:.*" {

          # 5. Login prompt received. Send user name to Unix server.
          exp_send "$username\r"
          exp_send_user "sent username\n"
      } eof {

          # 6. No login prompt received. Display an error.
          exp_send_user "could not connect\n"
      }

      # 7. Wait for a password prompt from the Unix server.
      expect "Password:" {
          # 8. Password prompt received. Send the password.
          exp_send "$password\r"
      }

      # 9. Wait for an FTP prompt. Enter FTP commands.
      expect -re $ftp_prompt {
          # 10. Change to the upload directory on the Unix server.
          exp_send "cd $target_directory\r"
      }

      expect -re $ftp_prompt {
          # 11. Change to the local directory you want to
          # upload from (Windows machine).
          exp_send "lcd $build_directory\r"
      }

      expect -re $ftp_prompt {
          # 12. Upload the file to the Unix server.
          exp_send "put $build_file\r"
      }

      expect -re $ftp_prompt {
          # 13. Close the FTP connection to the Unix server.
          exp_send "exit\r"
      }

1. Use a regular expression to match an FTP prompt.
2. Print this message to the console: "Connecting to [hostname] to upload the latest build".
3. An FTP session to a Unix server is opened from a Windows machine.
4. Expect waits for a login prompt.
5. The login prompt is received. The username is sent to the Unix server.
6. If a login prompt is not received, display an error in the Windows console.
7. Expect waits for a password prompt from the Unix server.
8. The password prompt is received. The password is sent to the Unix server.
9. Expect waits for an FTP prompt. Enter FTP commands.
10. Expect changes to the upload directory on the Unix server.
11. Expect changes to the local directory to where the source file is located.
12. The file is uploaded to the Unix server.
13. Expect closes the FTP connection to the Unix server.

Use Case 2: Updating Passwords

Scenario:
Use Expect for Windows to update a user's password on a Unix server via a Windows machine.

      # 1. Open a telnet session to the Unix server.
      spawn telnet $authserver

      # 2. Wait for a login prompt.
      expect -re ".*login: " {
        # 3. Send user login name.
        exp_send "$user\r"
      }

      expect "Password: " {
        # 4. Send the user's old password.
          exp_send "$oldpass\r"
      }

      expect -re ".*$ " {
          # 5. Run the passwd command on the Unix server.
          exp_send "passwd\r"
      }

      expect -re ".*password: " {
          # 6. Send the user's old password.
          exp_send "$oldpass\r"
      }

      expect "New password: " {
          # 7. Send the user's new password.
          exp_send "$newpass\r"
      }

      expect "Retype new password: " {
          # 8. Send the user's new password again.
          exp_send "$newpass\r"
      }

      # 9. Close the session.
      exp_send "exit\r"

      # 10. The Windows console displays a confirmation message.
      exp_send_user "Password for $user changed."

1. A telnet session to a Unix server is opened from a Windows machine.
2. Expect waits for a login prompt.
3. The user's login name is sent to the Unix server.
4. The user's old password is sent.
5. Expect runs the passwd command on the Unix server.
6. The user's old password is sent.
7. The user's new password is sent.
8. The user's new password is sent again for confirmation.
9. The telnet session is closed.
10. Expect displays a confirmation message in the Windows console.

Use Case 3: Remote Directory Lister Tutorial

Scenario:
Use Expect for Windows to automatically login to a remote Unix server from a Windows machine. Use the "ls" program to remotely list the contents of a directory. The Remote Directory Lister Tutorial demonstrates how to write a complete Tcl script using Expect for Windows. See the Remote Directory Lister Tutorial for more information.

Top

Porting Expect Scripts to Windows

The ActiveState Expect for Windows implementation differs from the original Expect written by Don Libes (at the National Institute of Standards and Technology (NIST)). These differences are due to Expect initially being written for the Unix platform. This section addresses the various command and behavior modifications in the Windows port. The last part lists the caveats you should consider when writing Expect scripts.

Expect for Windows Commands

To review all available commands and options, see the Expect for Windows Reference for more information.

Modified Commands and Behaviors

The following Expect commands and behaviors are modified for Windows usage:

Modified Commands:

• exp_close: The -slave option is not supported.
• exp_send_*: The -break flag is not supported. Does not implement a parser for VT100 codes for special keypress codes such as "arrow up", F1, Home, etc.
• exp_spawn: The -ignore and -trap options are not supported.
• exp_spawn: Supports two new options:
  
  • -environment: Takes a list of name/value pairs in the form that array get would return.
  • -directory: Sets the starting directory.
• exp_stty: Redirection is not supported. Row and column options are not supported.

Modified Behaviors:

• TTY channels are not created when not already provided to the application. This is not applicable on Unix. (For example, if CreateFile("CONOUT$") fails when asking for an existing TTY channel, there is no recourse in the generic layer to create one.) Only a single console window is allowed per process on Windows. This affects exp_send_tty and expect_tty, and is a prerequisite for enabling exp_interact.
• Signal manipulation beyond the seven signals abstracted via ANSI's signal.h is not supported, nor are windows-specific ones such as CTRL_SHUTDOWN_EVENT. Also, exp_wait is limited in the same way for signal exits, as is the core's Tcl_WaitPid() which it uses.

The following commands are not applicable to the Windows platform. When used, these commands only return errors.

• exp_fork: Not applicable on Windows.
• exp_overlay: Not applicable on Windows.
• exp_disconnect: Not applicable on Windows.
• exp_system: Use exec instead .
• exp_open: Detaching to Tcl is not enabled.

Unimplemented Commands and Behaviors

This section addresses those commands and behaviors that have yet to be implemented in Expect for Windows:

Unimplemented Commands:

• exp_interact: Usage creates a syntax error. Command is not implemented.

Unimplemented Behaviors:

• Expect clib: The C Library is not implemented.
• Expect channels: Cannot be moved to a different thread with thread::transfer. This is a limitation of Tcl's channel API.

Caveats

The following caveats should be reviewed before writing Expect for Windows scripts.

Extensions may collide with Expect command names. For example, send is defined by Tk for an entirely different purpose. This is why the commands listed have the "exp_" prefix. Use these extended command names to ensure compatibility between Windows and Unix environments.

Expect takes a liberal view of scoping. In particular, variables read by commands specific to the Expect program are sought first from the local scope, and if not found, in the global scope. For example, this obviates the need to place global timeout in every procedure that uses expect. On the other hand, variables written are always in the local scope (unless a "global" command has been issued). The most common problem this causes is when exp_spawn is executed in a procedure. Outside the procedure, spawn_id no longer exists, so the spawned process is no longer accessible simply because of scoping. Add a global spawn_id to such a procedure.

Terminal parameters can have a big effect on scripts. For example, if a script is written to look for echoing, it misbehaves if echoing is turned off. For this reason, Expect forces terminal parameters by default. Unfortunately, this can make things challenging for other programs. For example, the emacs shell wants to change the "usual" mappings: newlines get mapped to newlines instead of carriage-return newlines, and echoing is disabled. A user can use emacs to edit the input line. Unfortunately, Expect cannot possibly guess this. You can tell Expect not to override its default terminal parameters, but you must then be very careful when writing scripts for such environments. In the case of emacs, avoid depending upon things like echoing and end-of-line mappings.

Commands that accept arguments braced into a single list (the expect variants) use a heuristic to decide if the list is actually one argument or many. The heuristic can fail only in the case when the list actually does represent a single argument which has multiple embedded \n's with non-whitespace characters between them. This may seem unlikely, however, the argument -nobrace can be used to force a single argument to be handled as a single argument. This could conceivably be used with machine-generated Expect code. Similarly, -brace forces a single argument to be handled as multiple patterns or actions.

Tips for Writing Expect Scripts

This section addresses various tips for writing Expect scripts.

Tip 1: Use a shell command trick in your scripts to associate the file type on Unix systems.

Place the following commands at the beginning of your Expect for Windows scripts to run them in both Unix and Windows.

      #!/bin/sh
      # \
      exec tclsh "$0"  ${1+"$@"}

• In Unix, these instructions identify the file as a Tcl script. The exec tclsh "$0" ${1+"$@"} command launches the tclsh interpreter to run the Expect for Windows script in Unix.
• In Windows, these instructions are viewed as commented code and are ignored. Specifying an interpreter is unnecessary in Windows as the ".tcl" file extension identifies the file as a Tcl script (assuming that the PATH contains the Tcl interpreter).

Tip 2: Debug the process by viewing the controlled console.

Place the following variable near the beginning of your Expect for Windows scripts to view program interaction on the Windows platform.

      # This special variable used by the ActiveState Expect for Windows
      # port will enable actual viewing of the controlled console.
      # Otherwise it remains hidden (default).
      set exp::winnt_debug 1

• The set exp::winnt_debug 1 variable opens the controlled console and displays the interaction between the Expect for Windows script and the Unix program.

Tip 3: Use a regular expression to recognize shell prompts.

A common Expect usage challenge is determining which shell prompt to expect on the interacting Unix system. Since command prompts vary greatly among users and shells, automating Expect to interact with a program like rlogin can prove difficult.

One solution is to use a regular expression to match a variety of prompt types. Another solution is to use catch to see whether a particular user stored a regular expression describing their prompt in the EXPECT_PROMPT environment variable.

      set prompt "(%|#|\\$) $"    ;# default prompt
      catch {set prompt $env(EXPECT_PROMPT)}

      expect -re $prompt

• Use a regular expression to match a prompt, for example set prompt "(%|#|\\$) $"
• Attempt to catch the user's EXPECT_PROMPT environment variable.

To learn more about regular expression syntax and usage, see the Regular Expressions Primer for more information.

Tip 4: Matching output

About Output:
If you use a pattern of the form X*, the * matches all output received from the end of X to the last character received. This sounds intuitive but can be confusing because the phrase "last character received" can vary depending upon the speed of the computer and the processing of I/O, both by the kernel and the device driver.

In particular, humans tend to see program output arriving in large portions (atomically) when in reality most programs produce output one line at a time. Assuming this is the case, the *, in X*, may only match the end of the current line even though there seems to be more. This is because at the time of the match this included all received output. Expect does not know that further output is arriving unless your pattern specifically accounts for it.

Express the last few characters of a prompt:
Always express the last few characters of a prompt when writing patterns as depending on line-oriented buffering is unwise. Programs rarely reveal the type of buffering they do and system indigestion can break-up output lines at seemingly random places.

Use "timeout" and "eof" to determine last output:
If you are waiting for a pattern in the last output of a program and the program emits something else instead, you cannot detect it with the timeout keyword. The reason is that expect does not timeout. Rather, expect gets an eof indication. Use the eof indication along with the timeout keyword to be sure to located the true last output.

Matching a two-lined pattern:
Newlines are usually converted to carriage returns and linefeed sequences when output by the terminal driver. When matching a two-lined pattern as follows:

      printf("foo\nbar")

You may require a pattern constructed as follows:

      "foo\r\nbar"

Tip 5: Matching user input

A similar translation occurs when reading from the user, via expect_user. In this case, when you press "Return", the character is translated to a newline. If expect then passes that character to a program which sets its terminal to raw mode (like telnet), a problem results, as the program expects a true return. (Note that some programs are forgiving as they automatically translate newlines to returns, but most do not.) Unfortunately, there is no way to determine if a program puts its terminal into raw mode.

Rather than manually replacing newlines with returns, the solution is to use the command stty raw, which stops the translation. Note that you can not get the cooked line-editing features when stty raw is used.

Top

Wrapping Expect using TclApp

Use the Tcl Dev Kit's TclApp to seamlessly deploy your Expect for Windows programs on any platform without installing Tcl or otherwise configuring the system. The TclApp tool simplifies the process of distributing an Expect for Windows application by collecting all of the files needed to run a Tcl application -- such as Tcl scripts, Expect libraries, graphics and other data files, Tcl extensions, a Tcl interpreter, and the standard Tcl and Tk libraries -- into a single executable file. A user can then install the Expect for Windows file anywhere on their system and execute it without installing any other packages or reconfiguring their system.

To wrap Expect for Windows applications in TclApp, include the Expect base kit (located under Tcl/bin/). Use the Expect Tk base kit if you require the Tk package in your application.

• base-expect-win32-ix86.exe
• base-expect-tk-win32-ix86.exe

You can include an Expect base kit in TclApp via the command line or through the TclApp user interface. The Expect base kit includes the dll injector.dll which will be automatically extracted at runtime to the currently working directory of the wrapped base kit for the controlling of spawned applications. If this is not able to be extracted (e.g. on a read-only file system), and the injector.dll does not already exist on the system dll path, spawning will fail.

Command Line:

• Assign the base kit to a project using the -executable flag on the command line, for example:

      tclapp.tcl -nologogui -executable {C:\Tcl\bin\base-expect-tk-win32-ix86.exe}
        -out {C:\Tcl\tkremotels.exe} -pkg BWidget -relativeto C:\Tcl\demos\Expect
        C:\Tcl\demos\Expect\tkremotels.tcl

TclApp GUI Interface:

• Assign the base kit to a project using the Prefix file field on the Wrapping tab.

For more information about wrapping Expect scripts with TclApp, refer to the "tkremotels.tpj" demo. See TclApp on ASPN for more information on wrapping Tcl applications with TclApp.

Top