In the file download server behavior, you can set it to use the file_count column to increment each time the file is downloaded.
you need to create a recordset that checks the file count column:
SELECT * FROM table WHERE xID = 1 AND file_count <= 5
and use the Dreamweaver Show region behavior to show the download link if the recordset is not empty or show the error if the recordset is empty.