The implementation of a scan-trap depends on two assumptions:
First, that there are certain types of traffic that should never be coming in from the Internet - for example, SNMP, NFS (and related RPC protocols), and LPD. Many “back door” remote access trojans tend to listen on specific port numbers as well. Traffic destined to these services can, with a large degree of safety, be assumed to be hostile in all cases.
Note that given the propensity of unsophisticated Windows users to misconfigure their computers, inbound NetBIOS-over-TCP traffic should not be considered hostile - but don't forget to block it at the firewall!
Second, that most traffic destined to nonexistent hosts - IP addresses on the boundary network that are not in use - is the result of random scanning or scanning of entire blocks of network addresses. Such traffic should not, however, trigger an instantaneous response. Human error in misconfiguring an application or typing errors in URLs can generate traffic to nonexistent network addresses, and traffic to network addresses that were used in the past can persist long after the network address is no longer in use.
For purposes of illustration our boundary network will be 192.168.0.x
So, the first step is to generate a list of service requests that should be considered hostile whenever received from the Internet. Here is a starting point for TCP and UDP port numbers:
98 LinuxConf (system configuration, not from the Internet!)
111 Sun RPC (crackers scanning for Buffer Overflow flaws
permitting remote access, typically as root)
161 SNMP (BO flaws, insecure default configurations)
162 SNMP (BO flaws, insecure default configurations)
512 rexec (weak authentication)
513 rlogin (cracking weak passwords)
514 rsh (weak authentication)
515 LPD (BO flaws)
635 NFS mounts
1433 Microsoft SQL server (default install has blank admin password)
9704 various Windows back doors...
12345
12346
16959
22222
27374
31337
32771
32772
32773
32774
33270
34555
35555
54320
54321 ...my, there are a lot of these... :)
To keep this list up-to-date, you should monitor resources such as the SANS alert mailing list at http://www.sans.org/newlook/digests/SAC.htm, the CERT advisory mailing list at http://www.cert.org/contact_cert/certmaillist.html, and (of course) the BugTraq mailing list.
And the second step is to generate a list of service requests and unused boundary network addresses (or netblocks) that can be considered indications of hostile activity when taken together.
For the purposes of illustration we'll say that there are two large unused blocks of addresses on our boundary network: 192.168.0.128 - 192.168.0.159 and 192.168.0.192 - 192.168.0.223. If you get out your IP calculators, you'll see that those netblocks are 192.168.0.128/27 and 192.168.0.192/27. We'll assume that these addresses have never been used for any purpose that the public would know about.
For those netblocks we don't need to repeat the above hostile service list. Instead, we need to define services that are subject to attack but are more legitimately expected to be coming in from the Internet because there may indeed be such servers available to the public somewhere on the boundary network:
21 FTP (warez kiddies scanning for anonymous-writable servers)
22 SSH (BO flaws)
23 TELNET (BO flaws, weak passwords)
25 SMTP (spammers scanning for open relays)
53 DNS server (BO flaws)
80 HTTP (to catch HTTP worms scanning for vulnerable servers)
1080 SOCKS proxy (identity hiding, bounce attacks on others)
1723 PPTP server (DoS bugs, weak passwords)
3128 Squid HTTP/FTP proxy (DoS bugs, identity hiding)
3389 Microsoft Terminal Server (DoS bugs, weak passwords)
8080 HTTP proxy (identity hiding)
This traffic cannot, by itself, be considered hostile. To be considered hostile, it also needs to be destined to a computer that does not exist.