Have spent 4 hours being unable to disable toolbars using XML update only to find that the block is hard coded.  Found a fix here:

http://www.magentocommerce.com/boards/viewthread/46823

Then add this into category custom layout update

<reference name="content">
      <remove name="product_list_toolbar_pager"/>
</reference>
<reference name="content">
      <remove name="product_list_toolbar"/>
</reference>
<reference name="content"><remove name="product_list_toolbar_pager"/></reference>
<reference name="content"><remove name="product_list_toolbar"/></reference>