Although the overall concept of scientific publishing was omnipresent to me from early points in my studies onwards, there were multiple small details and not-so-small fundamental aspects I learnt when going through this process for the first time.

Expectations and trade-offs

A first basic aspect is the expectation of those that evaluate and receive the presented work, namely the peer referees at first and subsequently the readers. While constraints on time, personnel and computational resources are very much present when deciding the format and extent of published contents, the submitted manuscript is expected to cover a coherent set of the most relevant research questions, which is based on the nature of the subject and less on the available resources. This means that trade-offs between research questions have to be made by means of resource allocation, and that most likely a larger budget is necessary to fulfil all expectations. As a consequence, the outcomes of single steps (in my case numerical experiments) had an even larger influence on the trajectory and planning of the project.

This differs strongly from the expectation of referees for a (under)graduate thesis, which is evaluated under the awareness that a single person worked on it for a fixed time span and typically does not ask – or even allow – for extensions of any kind in order to optimally form a set of results. While the expectations and requirements in the publication process may put additional strain on the own timetable and budget, I experienced a related, more fundamental lesson when working towards the submission and resubmissions: The answers to single aspects of research questions did not – and to all I know will rarely – form a coherent, neat story on the matter in the way hypothesized at first. This required us to iteratively plan new experiments, adapt the questions we asked and reinterpret previous results and their importance in order to gain an overall understanding.

Of course this is a very important concept underlying the research process itself, but when we started working concretely towards the publication, evaluating our work from the perspective of a recipient and referee became more important and put additional emphasis on such ‘strategic’ aspects of the scientific process. In particular, I feel like this made me arrive at the right questions faster and keep an eye on the ‘big picture’ more consistently. Certainly, this may be typical when transitioning from an exploratory phase to the concrete setup of research questions, but I am sure this is a useful skill and habit in general.

When we started working concretely towards the publication, evaluating our work from the perspective of a recipient and referee became more important and put additional emphasis on such ‘strategic’ aspects of the scientific process.

The danger of the ‘streamlined story’

Overall the publication experience shaped my way of conducting research and – as it happens often – I already would have many things to tell my past self to improve on and which I try to implement in my current work. Despite these insights and their influence on both the research covered in my first publication and present work, the demand for a coherent, ‘streamlined’ story in scientific publications also raised some doubts in me: Whether the filter this demand puts on published outcomes might be too strong, encouraging researchers to sell their results as more monolithic than they are and to preferably pursue research questions with compatible answers. And whether the importance of negative results, in particular for young researchers and them constructing their intuition on research topics, might be underestimated both in scientific publishing and the community in general. I think it is very important to be aware of such incentives in the peer review process and to consciously avoid biases in our own research that may be caused by this.

Where do I click ‘hand in’?

Before receiving any feedback from peer referees, there were three tasks I did not have in mind while finishing up our paper: Choosing the journal to submit to, writing a letter to the editor and coming up with a collection of suggested referees.

While the first task is rather obviously necessary and I had witnessed quite some discussions about choices of journals, impact factors and the scientific publication press in general, I did not think about it until after the preprint submission. Our paper was well-suited for the journal we chose eventually, Physical Review Research, for the following reasons: First, we focused on evaluating proposed algorithms and their behaviour for different tasks and not so much on proposing a new algorithm or fundamentally new approaches to the topic. Second, in our methods section we aimed at making the background and the numerical setup understandable to a broad audience and at providing a complete picture of the task and the algorithms. This matches the scope of the journal very well. Third, all of us authors were in favor of making the paper openly accessible.

In hindsight, I fully understand why one writes a letter to the editor, but before the first publication I wasn’t aware of this concept.

I am glad that my supervisors decided to let me write it myself – with them providing some revisions – because this made me reflect on the scope of our work and consider the ‘big picture’ into which the paper would have to fit.

Lastly, I was very surprised that authors have the option of suggesting referees. While the idea of suggesting peers that might be most able to understand and review the work sounds good in the first place, I think that this is a danger for equal opportunities and fair career development. Typically, the suggested referees from the field will know articles beforehand because we use preprints, leading to biases in the peer review. The option to suggest referees favors researchers that are more established and already have larger professional social networks. In our case the journal has a wide scope across physics, such that suggesting referees within our field would not yield a representative review.

I am aware that the referee suggestions are what they are called – suggestions – and that the referees are chosen by the editor. Still I don’t think it is a good idea to introduce this bias and that the journal should be able to identify good referee candidates. We did not provide any suggestions.

On the bright side…

Regarding the publication process concretely, I gathered multiple, mostly positive insights:

First, the peer review provides valuable feedback both on the research and the chosen presentation. Independently of whether the publication was recommended or not, this helped me greatly to improve on evaluating research questions and on my scientific writing. As compared to the intense discussions and feedback from my supervisors and coauthors, the peer referees modeled a reader of the paper more realistically and mirrored their unbiased impression, even more so because we published in a cross-field journal.

Second, I was positively surprised by the referee reports. My expectations were rather low because of a plethora of stories, comments and jokes I had heard about the process and the stereotypical tone in the reports. These expectations turned out to be inadequate and as described above, I found the feedback very valuable and the quality of the paper improved drastically because of the review process – including a report that did not recommend our work for publication. The tone of the reports was very polite and professional, the correspondence was fact-based throughout and the level of detail surprised me very positively. I recall my first reaction to the report that discouraged publication to be very negative and emotional. In hindsight, I am sure that it was based on the rejection and that my negative expectations played a role in this as well, and I learnt that it is important to reflect my emotions towards my research and criticism.

Third, while I was positively surprised by the technical workflow for manuscript submissions and the integration with the arXiv preprint server, the final step of copy-editing was less fluent and seamless than I would have expected. In particular, the process could have been more transparent to save additional work and a software solution enabling authors to match journal requirements from the beginning might be possible and very helpful.

Overall, the publication process taught me many lessons regarding the particular project and our results but more generally regarding the scientific process, the presentation of ideas and results, and my personal relation towards my work.

About David

David likes to code and was just curious to check out these optimization techniques used in variational quantum algorithms. He’s still checking…

About the paper