The stamp can make sounds by hooking up an external, 8 ohm speaker to an amplifier controlled by a stamp pin. Here's the simplest way to do it.

Some tips:
To kill some of the high frequency hiss, you can try adding a 10 uF electrolytic capacitor as shown below (watch the orientation). Try with and without and see which you like.
Two Stamp commands generate sounds, freqout and dtmfout. For complete details, see the Stamp manual, available for free download from the Parallax web site.
Syntax:
freqout x, duration, note1, note2
where x = stamp pin (0-15), duration is length of tone in milliseconds, note1 is frequency (in Hz) of first note and note2 is that of the second which is played simultaneously to form a chord. The argument note2 is optional. For example
freqout 1, 1500, 500, 1000
uses Pin 1 to output a note 1.5 sec long that combines 500 Hz and 1000 Hz tones.
dtmfout x, [5,5,5,1,2,1,2]
is used to make the audio sound like telephone buttons. The example about uses pin x to "dial" the number 555-1212.
Try these two commands out and see what you get.
Here is some demo software downloaded off the Parallax web site that shows off sound capability (StampSounds.bs2).
Or, look up the freqout and dtmfout commands in the Stamp manual for more sample code.