By the Civil War, Castle Pinckney (built in 1810) was part of a network of defensive positions in Charleston harbor, which included the larger and more strategically placed Fort Sumter and Fort Moultrie. On December 27, 1860, one week after South Carolina seceded from the Union, the fort was surrendered to the South Carolina militia by its small garrison, which retired to Fort Sumter. Castle Pinckney became the first Federal military position seized forcefully by a Southern state government. After the subsequent attack on Fort Sumter, the Charleston Zouave Cadets manned Castle Pinckney. One hundred fifty-four Union Army prisoners of war captured during the First Battle of Manassas arrived at Charleston on September 10, 1861 and were kept at the Charleston City Jail until the lower casements of Castle Pinckney were converted into cells. According to the Charleston Mercury, Richmond officials had selected '...chiefly from among those who have evidenced the most insolent and insubordinate disposition.' On September 18, prisoners from the 11th New York Fire Zouaves, 69th New York 'Irish' Regiment, 79th New York Regiment, and 8th Michigan Infantry were transferred to Castle Pinckney. They were allowed to wander during the day and were confined to cells only at night. The Castle quickly proved to be inadequate for permanent confinement and the prisoners were transferred back to the Charleston after only six weeks. After the prisoners were removed, the fort was strengthened. On December 12, the prisoners were transferred back to the island following a fire which had burned a large section of Charleston and damaged the jail. They remained for just over a week with many sleeping on the inner parade ground before being transferred.