James Madison Caswell enlisted 11 Aug 1862 at age 25 as a Private in Company E, 13th Infantry. Mustered in 19 Sep 1862 Transferred to the Band where he played alto in the 2nd Brigade, 3rd Division, 24 Artillery 20 Jan 1863. He mustered out 21 Jun 1865 as a 3rd Class Musician at Richmond, VA. He resided in and was credited to Newmarket. He applied for a 
military pension on 13 May 1880.
He was born 11 Oct 1835 in Barrington to James and Lydia Caswell. He was married to Amanda Drew on 17 Mar 1856 in Barrington. (b. 8 Sep 1838 in Maine, d. 19 Dec 1886). They moved to Newmarket before 1862 when he mustered into the NH Infantry. They had two children, the first — Herbert J. (b.1863, d.1927) was born while James was away at war; their second — Charles (b.1866, d.1937) was born once James returned home. The Town directory of 1872 listed him as employed at a variety store, residing on South Street. In the 1880 census he describes his business as a “fancy goods store”. By 1910, at age 73 he was a Deputy Sheriff for Rockingham County. James died 18 March 1914 in Newmarket at age 77.
Published Newmarket Advertiser 7 April 1911
James M. Caswell has been reappointed deputy sheriff for Rockingham County. He is a veteran who is this year beginning his ninth term. At the end of the present term, he will have been in office for 18 years. Mr. Caswell has already served 16 years, or eight terms. He is in active health, after having been the sufferer from a broken hip several years ago, but this misfortune, with the exception of slight limp, does not affect his activity. He is one of the prominent citizens of Newmarket, and at one time engaged in business, being one of the merchants there for a period of 27 years. He is a member of the George A. Gay Post, G.A.R., being in the 13th New Hampshire regiment, and is also prominent in many branches of life in town.
One way a regimental band was formed (and by far the cheapest), was by drawing upon the resources available from among the men in each company. With ten companies to a regiment and two musicians allowed to each company—that is to say the fifers, buglers, and drummers—one could put together some kind of band of twenty men or more, if the officers agreed to detail to the regimental band musically qualified men who had not enlisted as musicians.
The 13th New Hampshire Volunteer Infantry is reputed to have the distinction of having the first U.S. flags in the city of Richmond, Virginia on April 3, 1865.
The following was written by James Caswell in 1886 for S. Millet Thompson when Thompson was gathering information for this Regimental History of the 13thInfantry.
“The Band of the Thirteenth was first organized at Concord, by selecting the two musicians belonging to each Company, which gave a membership of about twenty men to commence with. There were many changes. The Band remained with the Regiment until Jan. 20, 1863, when it became a Brigade Band, with the understanding that the Band should always remain in the same brigade with the Thirteenth. This was done in justice to the 13th, who had contributed about $700 for the original purchase of the instruments. The Band thus remained with the Reg. nearly all the time, and came home with it at the last.
“William M. Critchley was Band Master and Leader during the three years. Charles E. Graham served as Sergeant of the Band for the first year and a half; he received a
commission as Second Lieutenant in a colored regiment, and for the balance of the three years Charles W. Washburn served as Sergeant. Henry G. Parshley, by sonic considered the best musician in the Band, died of diphtheria. Henry Snow, Albion K. P. Shaw, and one or two others, were discharged the service because of sickness. A few members of the Band were supplied by other regiments in the brigade.“The Dirge most frequently played at funerals was the “Dead March in Saul”. We played that march through nineteen times at the funeral of Lieut. Sanborn, who was shot by
Dr. Wright of Norfolk.[*] His remains had been sent home several days before this funeral was celebrated. All the colored troops in the Department were present at the funeral and our Band was engaged to play by the First Regiment of U. S. colored troops.
(photo: emancipation Day, March into Richmond April 1865)
“The above items about the Band were furnished to the writer by Sgt. Chiles W. Washburn, who adds : ” We had the good fortune to be in the first Division to enter Richmond, and to blow our horns.”
“When the Band was mustered out of the service each man was given the instrument which he had used, and I have mine today “
Bands proliferated and, throughout the war performed on all manner of occasions, even during the heat of battle. The band of the 13thperformed in the trenches. Lieutenant Thompson of the 13th New Hampshire describes an incident occurring just after the battle of Cold Harbor, June 8, 1864:
“This evening the Band of the Thirteenth goes into the trenches at the front, and indulges in a “competition concert” with a band that is playing over across in the enemy’s trenches. The enemy’s Band renders Dixie, Bonnie Blue Flag, My Maryland, and other airs dear to the Southerner’s heart. Our Band replies with America, Star Spangled Banner, Old John Brown, etc. After a little time, the enemy’s band introduces another class of music; only to be joined almost instantly by our Band with the same tune. All at once the band over there stops, and a rebel battery opens with grape [**]. Very few of our men are exposed, so the enemy wasted his ammunition; while our band continues its playing, all the more earnestly until all their shelling is over.”
(Source: S. Millett Thompson, Thirteenth Regiment of New Hampshire Volunteer Infantry in the War of the Rebellion, 1861-1865: A Diary Covering Three Years and a Day (Boston: Houghton, Mifflin and Company, 1888), 369.)
[*], 11 Jul 1863, in downtown Norfolk, Lieutenat A.L. Sanborn, of Company B, Second United States Colored Volunteers, was marching his company down Main street on the sidewalk, a prominent citizen and violent Secessionist named Dr. DAVID M. WRIGHT, rushed from the store of FOSTER & MOORE, and shouted to the Lieutenant that he was a d — d cowardly son of a b — t. Dr. David Wright then shot him on the street. Lt.. Sanborn was from Vermont, formerly employed in the Quartermaster’s Department, Washington, and was very much beloved and respected by his fellow officers and the men under him. Wright was tried, convicted and hanged. Tensions in the City of Norfolk ran high all throughout the trial. Wright was described as a martyr by the Southern Press. This appeared in The Petersburg (Va.) Intelligencer after the hanging was carried out: “The family of the late Dr. D. M. Wright, of Norfolk. — says that any remittances in aid of the family of Dr. D. M. Wright, who died the death of a martyr at Norfolk, may be mailed to that office, and will reach its proper destination. The Intelligencer adds: Among all the afflictions brought upon families and individuals by this war, we have heard of no case which presents so many horrors as that of the family of the late Dr. David M. Wright, lately murdered by the Yankees in Norfolk. Since his cruel murder by hanging, his widow has died of distress of mind, and one of his daughters, a most interesting young lady, has gone mad. The eldest son of Dr. Wright having been killed at the battle of Gettysburg, his family are now left in indigent circumstances and without a protector.