What’s this all about?
Earlier this month I undertook a survey of experiences of MDPI among researchers. 1168 people took part. You can read the full report here, and download the data I collected here. You can read about the context of this survey – namely rapid growth in MDPI journals here.

How did you find 1168 people who wanted to talk about a publishing company?
I disseminated the survey on Twitter. The survey was only 3 minutes long and I sent it to my network. I also searched for people who had mentioned ‘mdpi’ in their tweets and replied to these tweets asking them to look at the survey. It made quite a hit, over 350 people retweeted the survey, and the interest was such that MDPI also promoted it too.

But if people are talking about MDPI on Twitter then they are either celebrating their papers and Special Issues, or complaining. Doesn’t your method simply produce polarised results?
Well it didn’t in this case. There were lots of more ambivalent responses. I guess because when people retweeted then that brought in a network of less strongly held opinions. But you are right to be cautious. We cannot assume this survey is representative of the research community.

Not representative? So what’s the point of it then?
It identifies patterns and associations in views about MDPI journals, and tells us what features different groupings share. A more systematic sampling method can determine how common these groupings are.

I have a low boredom threshold and you’re reaching it. Give me the answer quickly.
There is no single answer.

Snore
But one of the more interesting findings . . .

SNORE
. . is how important responses to email communications are for people’s thoughts about the journals. Basically, lots of people are receiving a large variety of email from MDPI and many do not like it. You can see from the table below that the ‘communication appreciation index’ is pretty low for people who received three or more types of email – and it was quite hard to find people who received few types of invitation.

Table 1: Average Communication Appreciation Index and the Variety of Invitations

Career Stage	0-2 types of Invitation to engage with MDPI	3+ types of invitations  to engage with MDPI
Junior	-1.3	-3.9
Mid	-0.3	-4.6
Senior	1.0	-3.1
	No. of Respondents	No. of Respondents	Total
Junior	141	359	500
Mid	46	364	410
Senior	28	229	257
Total	215	952	1,167

Scores in the Appreciation Index range from -10 (no desire to receive any communication) to 10 (wants to receive every communication)

Are you telling me that you asked 1168 people if they liked being spammed? Do you know what the word ‘research’ means?
I did not ask them that. But this result is important because even people who liked the company most did not always appreciate receiving so many emails. You can see that in the box of free text comments below.

Box 1: Issues with email raised by MDPI Enthusiasts

Too many emails, it is like spam. It is like a predatory journal in this regard. Some interesting journals and publications, but some politics should change.   I have worked with one that is among the stronger of their journals . . . But, I get a number of requests from MDPI from journals that are not even in my area of expertise. So some requests for our work do not seem well thought out.   They are relatively cheap and it’s easy to publish in them . . . But I hate the nagging, the angry emails because on Monday you haven’t replied to the email they sent on Saturday . . . I hate being asked to review irrelevant and often incomprehensible papers. Way too many unsolicited emails.

Too many invitations for serving as guest editors on special issues and submitting to special issues . . . While l like MDPI’s publishing speed and model, these kinds of practices lowers its reputation in my (and others’) minds. The publisher is fine – it’s those annoying e-mails that really tarnish their reputation. Reviewed many times in the past for them, published with too. But became too much of a nuisance . . . Have asked them to take me off reviewer list, which they ignored and continuously hound me. Shame – as good journal’s with strong academic side. They send me lots of email asking to edit a special issue. Lots!

But this box leaves out all the comments of people who are much more fed up, who aren’t ‘enthusiasts’ – why are you silencing them?
Qualitative research does that I’m afraid. The data we leave out can be as telling as those we include, and we always leave out a lot. I’ve put this group’s comments in because it was surprising for me that people who were enthusiastic about the company said things like this.

What do you mean by ‘enthusiast’ anyway?
That’s the second main finding. I was able to group respondents into five basic categories. There were those who are Hostile to MDPI without ever having engaged with it. Others seem to be Put-Off by their engagements with the journal. Some are Ambivalent about what they will do in the future. They are not sure if they will get involved or not. Then there are those who are Engaged and are definitely going to submit papers. The Enthusiasts are those who want to take on editorial duties.

And how do you know that these categories actually exist?
Well I asked people what they had done and will do with the journals. And the groupings that emerged had clear differences in terms of their communication appreciation indexes and brand index scores. Here’s a table with all the differences laid out and explained.

What are all these ‘indexes’, did people reply to you in numbers?
No, an index is just a way of converting categorical responses into numbers. If someone said, ‘yes I want to be invited to edit a Special Issue’, I might score that as ‘10’. Negative responses I’ll score as ‘-10’. Then I take the average. It’s all explained in the Appendices to the full report.

And can you explain that without sounding patronising?
Not really, no. I’m just not very good at that sort of thing. My daughters forbid me to talking to their friends, and just about everyone else, as a result.

I like your daughters. I thought good researchers were meant to produce decent pictures. Where are yours?
Well here’s a graph of how different disciplines responded to MDPI’s brand associations with ‘Rigour’ and ‘Importance’. Medics, Engineers and Humanities Scholars tended to Agree that rigour and importance were associated with the MDPI brand. Natural and Social Scientists tend to Disagree.

So what we think about the journals is decided by our discipline?
No. Discipline matters, but what we think about them is decided by our views on their rigour, importance and prestige etc – all things that I asked about in brand associations, and by our appreciation of communications. I conducted an ordinal logistic regression to tease out the exact relationships, which is included in the main report.

I bet you’re proud of that.
‘Ordinal Logistic Regression’ is hard to spell, let alone understand.

And were you able to do anything useful with this survey, like sort out whether the journals are well-reviewed or not? 40 days submission-publication is short.
That is short – and clearly deliberately so. A number of respondents emphasised how convenient and appealing that was. But of course there is no clear answer to this issue. I was told by some people about their excellent, rigorous – and fast – peer review experiences, and other people said that they were disappointed by it. I’ve included a selection of the comments below.

Box 2: Experiences of Review, with Respondent’s Engagement Category shown.

Poor Review Experience

Good Review Experience

Review process can be somewhat random, with editors and reviewers without enough expertise. (Enthusiastic)   I had a good experience with two of them and they are deemed to be reputable in my field. However, the quality of peer review and published papers is questionable for three other journals I was invited to write for or to do a peer-review. (Enthusiastic)   I published 2 papers, both had good reviews, but the whole experience seemed less rigorous than with other journals. The emphasis of peer review seemed to be on a box checked rather than improving the paper. (Engaged)   I was pleased by the efficiency and simplicity of publishing with them. I did however find the review process very hasty, with the reviewers providing very little meaningful comment. The editor seemed fine with this and did not raise any issues. (Engaged)   Reviews do not seem to be done by researchers. I submitted a paper and 3 out of 5 reviewers were focused on the paper format (e.g. reference style). (Ambivalent)   I have had normal papers accepted. The reviewers were poorly chosen in most cases, and the reviews were poor (short, lack of detail…). (Ambivalent)

Submitted only two articles so far. So not much to say, except got some lengthy four reviews for one article that was rejected after major revisions, as two reviewers were not satisfied with revisions. (Enthusiastic)   One paper was submitted there and received 4 reviews – total 17 pages of comments! It took lot a of time to revise and respond to reviewers comments. One of the reviewers disclosed their identity and was an expert in that field. It was accepted after two round of revisions. (Enthusiastic)   I have had good experiences with MDPI. Fast and rigorous peer review (four reviewers, excellent and challenging reviews, 10 days after my submission). (Engaged)   So far in my 2 first author papers I’ve had 4 reviewers each time and they were generally competent. There is always a “reviewer 2” but I’ve been happy with the rigour so far. (Engaged) I published with them as an early-career researcher because they offered to waive open-access fees which for an ECR is rather convenient. It did go through a fairly thorough peer review (2 rounds, 6 reviewers in total). (Ambivalent) I had a good experience with my one and only paper published with them. 4 reviewers, provided useful and prompt reviews. (Ambivalent)

So it’s just a case of separate academic tribes then. We find the people we agree with about MDPI journals and hang out with them.
No. You’re missing the point that these groupings – these tribes – are changing and evolving. On the one hand MDPI is growing. More and more people are publishing with them. On the other hand that very success is creating tensions and strains. It’s producing unwelcome email requests. And the increased speed is worrying some respondents. There was a sense in the survey from some that they did not like the way things were heading. After all, peer review is meant to work all the time, not most of the time. I’ve reproduced a selection of those views below.

Box 3: Changing Opinions about MDPI over time

In my perception, MDPI overall has a neutral or even slightly negative reputation. MDPI journals are very heterogenous, with the quality of journals depending a lot on their respective EiCs and Editorial board members. I am the [redacted] of an MDPI journal, try to keep rigorous standards, and regularly have to deal with negative spillovers arising from problems with other MDPI journals. (Enthusiastic) MDPI could improve its reputation by cutting back on the number of journals and special issues (so that special issues are actually special), avoiding sending irrelevant invitations and other predatory practices. The poor reputation of MDPI as a whole is damaging the more reputable MDPI journals that are trying to publish quality research while also offering authors a good service. (Enthusiastic)   I have edited two special issues and served on the editorial board of [redacted]. I stepped down in [redacted] because I felt uncomfortable about the very rapid increase in no. of associate editors, no. of papers published, no. of special issues. (Engaged)   MDPI . . is spamming special issues, some of which do contain very good papers. My own experience as reviewer was good so far, and as author mixed. I wouldn’t consider it predatory yet, but it is moving in that direction . . MDPI is rapidly degrading its reputation and the sympathy they had in my field as OA-publishers at the moment. (Engaged)   I strongly disagree with the MDPI policy of launching innumerable special issues. I have refused many invitations to edit special issues (I do not even reply to these invitations), these are really a nuisance for me. (Engaged)   Although my experience in publishing and reviewing for MDPI has been good overall, news and opinion by people in academia that it is a predatory publisher (despite some journals being really good and some still being not and have room to improve) can affect the view of my institution and superiors too, and may not allow us to publish in MDPI journals as it affects the reputation not just of the institution but affects my performance evaluation too and so recently I opt not to publish or review in MDPI journals. (Engaged)   When I started my PhD and then transitioned to a postdoc, my opinion on MDPI journal was very good. I liked the open access part of it and that papers were published quickly . . . [it] was a just fantastic. Then I was invite to review a paper, then another one, then the list grew. Then I became a guest editor, then a member of the editorial board. This allowed me to see MDPI from the inside and I quickly realized what their real priorities are. Everything is done as quickly as possible, everything has a short due date, and everything is a part of a special issue so that the guest editors do all the work. For free, of course. I do not regret having my papers published in MDPI before, and do not say I will not submit my papers to MDPI again. However, their journals will never be my first choice. Further, every time I read a paper published by MDPI I am extra careful and suspicious of the quality of the presented research. (Ambivalent)

So the growth won’t continue?
There is no way we can conclude that from this survey. There are cracks appearing. But we don’t know how wide and deep these cracks are. We don’t know what patterns like this look like for other publishing houses. And MDPI journals are incredibly varied too.

Remember that people have been worried about MDPI’s practices for some time, and growth has continued despite that. Paolo Crosetto has argued that, because of recent changes, growth is now not sustainable (here), and Volker Beckmann has replied (in a comment to that blog) that the growth demonstrates the high quality of service these journals provide, and therefore will be sustainable.

However the difference now is that a few years ago not many people had published with MDPI and not many people had heard of them. But the growth in the number of authors has been so great (see Table 2 below) that this is no longer true. I’ve discussed this in a previous blog which explores trends in growth of MDPI journals from 2015-2020. The scale and speed of MDPI’s success means that reputational issues simply matter more. My hunch is that MDPI will have to be more careful about its emails, and possibly slow down MS processing in some journals.

Table 2: Authors by Region of their Institution in MDPI Journals

Region	2016	2017	2018	2019	2020
Aus-Pac	1,077	1,629	2,747	3,914	5,852
C Asia	197	492	1,128	2,518	5,309
E. Asia	13,432	20,982	39,323	65,128	77,389
Europe	9,744	15,906	29,988	56,346	103,155
L Am, C	918	1,728	3,174	5,441	8,701
MENA	756	1,112	2,111	4,279	8,060
N. Am	5,112	7,606	12,156	17,657	25,975
S. Asia	412	685	1,306	2,487	3,462
SE Asia	623	980	1,754	3,131	5,618
SS Afric	372	608	865	1,386	2,516
Total	32,643	51,728	94,552	162,287	246,037

Source: Country level data provided by MDPI

Are you ever going to form a definitive opinion about anything? Have you just asked 1168 people to fill in a survey so that you can say ‘errrm’?
You’re very clear minded. Please do analyse these data to see for yourself what the patterns are. And please look at the full report to get into the nuances. You know you really want to. And then you can tell me where I need to be more decisive.

There’s a lot we’ve learnt from this survey. The overabundance of email is one thing, and that suggests it will be useful for Editors-in-Chief to monitor the ‘hit rate’ of email requests their journals send. If these start going down, then clearly people are disengaging from their journal. We’ve shown how important discipline can be, and how distinct academic communities may sustain, and be sustained by, different journals. This will help sampling strategies of future research. And we’ve shown that brand reputation can bleed across the journals. Which means that journals’ leadership may need to take an interest in each other’s activities (like review glitches or email hit rates).

And finally, remember that MDPI journals are part of a broader publishing ecosystem. We have to consider our responses to these journals within that wider context. I think this survey prompts a series of questions for different groups of people – MDPI staff, Editors, Authors, Reviewers, Critics. A companion blog presents these questions.

My thanks to all who have taken part in this survey and spread the word about it.

Dan Brockington
Glossop
19^th April 2021

The full report on the survey is available here.
The data collected for the survey are available here.