Pages in topic:   < [1 2]
Some issues with how grading is handled
Thread poster: Łukasz Gos-Furmankiewicz
Phil Hand
Phil Hand  Identity Verified
China
Local time: 15:52
Chinese to English
Why no voting in competitions? Aug 28, 2014

Orrin Cummins wrote:

Merriam Webster...fair and equal chance... Peer-grading is inherently contrary to that ideal.

Genuinely can't see what the issue is here. Certainly, expert judging is one model of how contests can be run. But peer/audience voting is another. Think of those karaoke shows they have now, American Idol and X Factor. The audience vote in at least some of those (and presumably the contestants can, too, if they have a mobile phone). I don't see why that would invalidate them as contests. It certainly changes the character of the contest a bit, but it's still a contest.

So what you have then is a contest operating on the honor system.

Not sure how much honour is required. Each member gets only one vote. In any pair with a reasonable number of entries, one vote is unlikely to be very decisive.

I agree that it makes little sense to monolingually judge one side of a translation

Apparently it's "Orrin is wrong day" for me. This again appears to be a massive whopper. Everyone in the world other than translators judges translations monolingually! That's the whole point of us. We produce stuff for people who can't read the source language. So monolingual judging could be said to be *the most* relevant way to judge our output.

Obviously I get the argument, looking at the target text only allows for no assessment of fidelity. Fine. But as I said above, translation is an inherently plural activity, with multiple modes, multiple kinds of success, and multiple skill requirements. I'm much more suspicious of competitions which claim to be making singular, absolute judgments of translation quality.


 
Orrin Cummins
Orrin Cummins  Identity Verified
Japan
Local time: 16:52
Japanese to English
+ ...
OK Aug 28, 2014

Guess we'll just have to agree to disagree, then. No big deal.

I have yet to see the Japanese to English pair in a contest, so my opinion may be moot anyway (as is usually the case). I'll leave the debating to those members who actually have a dog in this fight.

But I do want to ask you something. You said:

We produce stuff for people who can't read the source language.


This is of course true. But if I am a painter who spends all his time creating art commissioned by blind people, does that necessarily mean that if I enter an art contest my painting should not be judged by other artists--or at least other people who can see?

In the real world, it is as you said--but in the real world, there aren't usually multiple translators who translate the exact same source text then compete against each other. The dynamic there is a little different than the translation contests here on ProZ, I think.

Perhaps I should have said:

I agree that it makes little sense to monolingually judge one side of a translation in a contest where the whole point is assessing the quality of that translation vis a vis other translations of the exact same source text.


 
Phil Hand
Phil Hand  Identity Verified
China
Local time: 15:52
Chinese to English
You see exclusion, I see inclusion Aug 28, 2014

Orrin Cummins wrote:

Guess we'll just have to agree to disagree, then. No big deal.

True dat.
But if I am a painter who spends all his time creating art commissioned by blind people

Allow me to interject: WTF?!
does that necessarily mean that if I enter an art contest my painting should not be judged by other artists--or at least other people who can see?

I don't get where this exclusion worry is coming from. The competition *is* being judged by other artists. I haven't gone and looked in detail at the rules, but I remember the last time I entered the English-Chinese competition, the vast majority of votes were cast by English-Chinese translators. Is it really the case that French translators are swooping in and messing up the voting in the Japanese-English pair?

Lukasz and you seem worried that allowing other options for commenting/grading will exclude or drive out or dilute the obvious main pool of graders. I completely accept that the obvious people to grade a German to Russian translation are German to Russian translators. I'm just saying, I see no signs that these people are being excluded or driven out by allowing other options.

Perhaps I should have said:

I agree that it makes little sense to monolingually judge one side of a translation in a contest where the whole point is assessing the quality of that translation vis a vis other translations of the exact same source text.

Once again, I have to disagree. Access to multiple translations is one of the ways in which monolingual readers *do* get to find out about misreadings of the source. And if you've got a bunch of translations which are faithful to the source - which I'm sure we have - then in fact the *only* criterion for distinguishing between them is their quality as pieces of writing in the target language.


Lukasz raises a good point:
I believe that good writing is a bad translation of bad writing.

There are a couple of responses to this. Firstly, often it's simply untrue for commercial translation. Clients expect readable texts, and they are not impressed if we say, "There was a gender disagreement in the source, so I wrote a bad sentence in my translation."

In translation of creative writing, where it is certainly fair to argue that we are trying to precisely convey the quality of the original, one might concede the point, but say this: in reality, the vast majority of translations are written worse than their source texts. The difficulty of following source meaning in target idiom ensures it. Therefore, in general the problem for the translator is how to improve the quality of her writing, not how to reduce the quality. Therefore in translator education (and I would argue that a competition is a form of ongoing education), it is right to emphasise good writing.

Not knock-down arguments, either of them, but I think the implications of Lukasz's idea are much harder than the simple assertion suggests. If you demand from every translation an assessment of the quality of the prose in the original, and then construction of prose of precisely the same level of quality in the target... well, that's a lot of hard work, and every step of the way is deeply controversial. Moreover, I think the problem that you are alluding to - all these brilliantly-written translations which get the meaning wrong - simply doesn't exist. The world is overflowing with ugly translations, not beautiful but inaccurate translations.

(Possible exception - there is a savage review of Mo Yan out there which suggests that his Nobel is entirely down to his translator. I have read very little of his, so I reserve judgment.)


 
Łukasz Gos-Furmankiewicz
Łukasz Gos-Furmankiewicz  Identity Verified
Poland
Local time: 08:52
English to Polish
+ ...
TOPIC STARTER
... Aug 28, 2014

Phil Hand wrote:

Lukasz raises a good point:
I believe that good writing is a bad translation of bad writing.

There are a couple of responses to this. Firstly, often it's simply untrue for commercial translation. Clients expect readable texts, and they are not impressed if we say, "There was a gender disagreement in the source, so I wrote a bad sentence in my translation."


That's their problem, Phil. Does a lawyer care whether his client is impressed with what the law is, or does a doctor care whatever diagnosis a patient expects?

There is nothing wrong with ordering translation + editing, where editing is a separate service or an optional component which expands the definition of the service.

The paradigm of modern translation, however, seems to be Stockholm Syndrome. Bullied translators act as false mirrors in offering flattering translations and ditching the professional and academic considerations of their art and science in favour of pleasing their masters.

In translation of creative writing, where it is certainly fair to argue that we are trying to precisely convey the quality of the original, one might concede the point, but say this: in reality, the vast majority of translations are written worse than their source texts. The difficulty of following source meaning in target idiom ensures it. Therefore, in general the problem for the translator is how to improve the quality of her writing, not how to reduce the quality.


I was going to agree when I started typing, Phil, but the reality is that many writers are quite sloppy people who expect their editors and proofreaders to fill in the punctuation. There is little reason to expect tenses, numbers, genders etc. to have been handled better. That's still fine as long as the translator receives a completed source. What is not fine is when clients supply rough drafts in the source language and hold the translator accountable for delivering 'final quality' regardless of the condition of the source — that's monolingual grading, in abstraction from the source. It is obviously very convenient to corporate clients without too much ethical sensitivity. They'll just redefine the problem so that it's the translator's obligation to make sure the effect of translation meets certain criteria — or is explicitly the best possible — regardless what the source looks like. Shameless use of mental manipulation, even on the ethical plane, is nothing unusual in business.

Commercial or not, monolingual grading promotes just that.

The other thing that monolingual grading does is completely flatten any sort of individual expression, idiolect, idiosyncratic features. Any quirk or oddity will be blamed on the translator. This is similar to monolingual review by native speakers who don't know the source language too well but are aware that the translator wasn't a native speaker of the target language: they'll keep second-guessing the translator to the point that only the most basic, standard syntax and grammar will survive. Anything else will be presumed to be an error or a flaw in the translator's writing style and sometimes 'corrected' in a way that alters the meaning. This is similar to the representativeness heuristic.

Even when the target-native person is supposedly restricted to grading the quality of writing only, the same cognitive distortions will make the whole thing a joke much or even most of the time.

Therefore in translator education (and I would argue that a competition is a form of ongoing education), it is right to emphasise good writing.


Of course. But only in a fair, honest and straightforward manner, without obfuscating what's really the translator's job and what translation is about.

Not knock-down arguments, either of them, but I think the implications of Lukasz's idea are much harder than the simple assertion suggests. If you demand from every translation an assessment of the quality of the prose in the original, and then construction of prose of precisely the same level of quality in the target... well, that's a lot of hard work, and every step of the way is deeply controversial.


True. But if the assessment is so difficult, making it simpler at the cost of veracity (i.e. devising and using a simplified heuristic) is not the only answer available. The alternative is asking ourselves whether we really need that assessment to take place, whether there are any other ways, whether it's worth it at all.

As far as a simplified assesment goes, anyway, it's really hard to get any sort of assessment accuracy without being able to understand the source (even if detailed analysis would be omitted to save time and conserve resources).

Moreover, I think the problem that you are alluding to - all these brilliantly-written translations which get the meaning wrong - simply doesn't exist. The world is overflowing with ugly translations, not beautiful but inaccurate translations.


I wouldn't be too quick to agree with that, Phil. Reviewers often complain about what they see as flaws in the translation without referring to the source text to check. Translators are aware of that, and it can't not affect how they work.

Plus, the world is not totally void of translators who attempt to fix a correct source and get it all wrong (legal translation especially). Of course, in such a case there is also the underlying problem of the limitations of their knowledge, skill and perception or some problems with their attitude.

... But monolingual review itself — reliant on simplified heuristics so much that it simply can't be considered reliable — is one of the huge attitude problems of modern translation. I mean review as in a sort of judgement here, I'm not talking about using non-speakers of the source to simply contribute another pair of eyes before releasing the final version.

(Possible exception - there is a savage review of Mo Yan out there which suggests that his Nobel is entirely down to his translator. I have read very little of his, so I reserve judgment.)


I can already see practicing translators and translation theorists cooing over that and exalting it as an example of what translation really should be or at least what it should aim for. And that's similar to the omnipresent naive translator marketing strategy that puts emphasis on 'added value' explained in terms of how translators help fix bad sources for their clients. 'Oh please hire me because I can and am willing to take over the burden of getting someone else's job right without asking for commensurate pay or anything more than a pat on the back, but even that's optional.' That's how a cowed, broken dog acts, not a human person more or less conscious of his own value. If you and the others will excuse my French.

***

Bottom line: there are two factors here, neither of which is critical on its own:

1. monolingual review;
2. emphasis on target writing quality as an autonomous or fully independent level of translation.

It gets really bad when #1 meets #2.


 
Pages in topic:   < [1 2]


To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Lucia Leszinsky[Call to this topic]

You can also contact site staff by submitting a support request »

Some issues with how grading is handled






Protemos translation business management system
Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!

The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.

More info »
Anycount & Translation Office 3000
Translation Office 3000

Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.

More info »