After reading your post here I thought I had found a good solution for you. Then I read your post on the Bookends forum which is somewhat different. It seems you need not only the *bibliography* to be sorted in a specific way (which isn't a problem) but the *endnotes* should also be sorted in a specific way (which isn't a problem either); but then you want to use inline citations as well, together with endnotes, which is problematic. I'm not saying it's impossible, but usually people have EITHER inline citations OR endnotes, not both.
Since I don't know which one you want, I will post here two
solutions for you, one for endnotes and another for inline citations. If you follow the description closely you'll see that this can be easily done. Don't let my verbosity put you off. :–)
"Automatically or manually reorder endnotes", as you say, is not possible because it makes no sense. Endnotes keep track of where reference symbols appear in the document. Instead
(1) create a new, second note stream and name it "Archival Source". Superscript one of the note symbols so the groups will be easily distinguishable.
(2) Choose "End of Document" as Position for this new stream. Use this stream exclusively for sources.
(3) Use another endnote stream for literature. This will automatically create two separate groups as endnotes.
(4) Next, you need to create a format in Bookends
which captures archival sources. Open the Formats Manager in Bookends and create a new Reference Type (or duplicate an old Type) in your favorite format and call the new type "Archival Source". You may need to create more than one type, depending on whether your sources are books, articles, manuscripts or whatever.
(5) In the Formats Manager, select the tab "Citation Options" and enter into the "Field Order"-field how you want the sources to be output in the endnotes.
(6) Then, while the "Citation Options" window is still open, make sure you have chosen "Archival Source" in the Type dropdown-menu.
(7) Then click on the Common Settings tab and enclose citations with "Nothing".
(8) When you scan your document choose "Generate a Subject
Bibliography". You will be prompted to choose a subject on which the Subject Bibliography should be based. Choose 'Reference Type' from the Subject menu and select the pertaining Reference Types. Then click OK. WARNING: if you don't select anything, the OK button is grayed out.
(9) What you should see now is a "Subject Bibliography" consisting of and sorted by the names of your reference types, i.e. Archival Source, Book, Edited book, Journal article, etc.
(10) Below the bibliography your endnotes are displayed in two separate groups.
(11) All you need to do now is to change the source numbers in the bibliography manually so they match
the order in the endnote part.
(12) When that's done, delete all subject titles (= reference types) in the bibliography other than "Archival Sources."
(13) At the end of "Archival Sources" insert one or two empty new lines and type "Literature", or whatever you find appropriate for this part of the bibliography.
(14) In this part of the bibliography (= the literature part) each reference is still preceded by an opening bracket, a number, a closing bracket and a tab, like this: 
(15) Select all the references in this part of the bibliography and delete the unnecessary numbers and brackets with a Find & Replace action.
(16) Then sort references in this part of the bibliography alphabetically, and you're done.
Here are screenshots which show the settings in Bookends for the endnote solution:
https://www.mediafire.com/file/38fm5qc3 ... S.zip/file
Solution TWO (for inline citations) follows in a separate post.