Create a calculated field for the Holidays using the date in your data.
Holidays = IF [Date]=<Holiday Date1> THEN 1 ESLEIF [Date]=<Holiday Date2> THEN 2 etc... ELSE 0 END.
Make it discrete.
Drag this field in the Color shelf and select colors for all the values and 0 will be for all the other values.
If there is already a field in color, then first Drag the Holiday field in Details and then change it to Color.