Comments for IREvalEtAl http://blog.codalism.com William Webber's Research Blog Fri, 27 Aug 2010 02:57:11 +0000 http://wordpress.org/?v=2.8.1 hourly 1 Comment on Computer science is not real science by Faskabot http://blog.codalism.com/?p=938&cpage=1#comment-14755 Faskabot Fri, 27 Aug 2010 02:57:11 +0000 http://blog.codalism.com/?p=938#comment-14755 I like how you completely ignore Theoretical Computer Science in your post. TCS is definitely Mathematics, and not "Mathematical Engineering". It aims to study and research the deep underlying patterns of the mathematical entities of computation to find out new theorems and properties, not merely "applying mathematical tools to serve application purposes". The mathematical entities studied range from algorithms, where computer scientists reduce their properties to discover similarities and qualify different algorithms under different algorithmic classes according to the way said algorithms tackle a problem; all the different kinds of automata, to discover numerical, combinatorial and computational properties of language processing automata, such as mathematically analysing and discovering the computational possibilities of different classes of automata, as well as the qualification of different languages according to the kind of automathon that can compute them, and other theorems regarding the nature of automata themselves (such as the theorem that states that for any NFA an equivalent DFA exists); computational complexity classes, discovering different properties that make problems belong to certain classes under a certain computational model, but tp other complexity classes under other theoretical computational models, discovering the patterns and properties of different computational problems situated in ranges of their complexity classes (an important theorem in this subject is Shaefer's dichotomy Theorem); and finally, the study of mathematical abstractions of computational models, including, but not limited to turing machines, quantum computers and other theoretical non-deterministic impossible "oracles". By saying that "the computational world is already mathematically described" you're completely taking for granted the structures cited above, which are studied and researched by Computer Scientists to reduce new conclusions about the immutable properties surrounding computation in our reality. It might be that algorithms (mathematical imperative statements) existed for a long time already, but an in-depth study on their true nature was never done before Alan Turing, who created Theoretical Computer Science. Automata, complexity classes, computational classes of combinatorial languages and the properties of theoretical computational models are all mathematical structures that had to be reduced and researched by computer science, and they have their immutable properties, theorems and definitions, just like any other mathematical entities, not merely "inventions to serve a purpose", but actual "mathematical structures reduced by the study of the underlying properties of computation". So, in fact, as you can see above, there was A LOT that still had to be reduced in computer science. You might argue that this all is mathematics, not science, but nor is it engineering. tl;dr version: Stop being so narrow-minded. I like how you completely ignore Theoretical Computer Science in your post. TCS is definitely Mathematics, and not “Mathematical Engineering”. It aims to study and research the deep underlying patterns of the mathematical entities of computation to find out new theorems and properties, not merely “applying mathematical tools to serve application purposes”.

The mathematical entities studied range from algorithms, where computer scientists reduce their properties to discover similarities and qualify different algorithms under different algorithmic classes according to the way said algorithms tackle a problem; all the different kinds of automata, to discover numerical, combinatorial and computational properties of language processing automata, such as mathematically analysing and discovering the computational possibilities of different classes of automata, as well as the qualification of different languages according to the kind of automathon that can compute them, and other theorems regarding the nature of automata themselves (such as the theorem that states that for any NFA an equivalent DFA exists); computational complexity classes, discovering different properties that make problems belong to certain classes under a certain computational model, but tp other complexity classes under other theoretical computational models, discovering the patterns and properties of different computational problems situated in ranges of their complexity classes (an important theorem in this subject is Shaefer’s dichotomy Theorem); and finally, the study of mathematical abstractions of computational models, including, but not limited to turing machines, quantum computers and other theoretical non-deterministic impossible “oracles”.

By saying that “the computational world is already mathematically described” you’re completely taking for granted the structures cited above, which are studied and researched by Computer Scientists to reduce new conclusions about the immutable properties surrounding computation in our reality. It might be that algorithms (mathematical imperative statements) existed for a long time already, but an in-depth study on their true nature was never done before Alan Turing, who created Theoretical Computer Science. Automata, complexity classes, computational classes of combinatorial languages and the properties of theoretical computational models are all mathematical structures that had to be reduced and researched by computer science, and they have their immutable properties, theorems and definitions, just like any other mathematical entities, not merely “inventions to serve a purpose”, but actual “mathematical structures reduced by the study of the underlying properties of computation”.

So, in fact, as you can see above, there was A LOT that still had to be reduced in computer science.

You might argue that this all is mathematics, not science, but nor is it engineering.

tl;dr version: Stop being so narrow-minded.

]]> Comment on Computer science is not real science by Gretgor http://blog.codalism.com/?p=938&cpage=1#comment-14439 Gretgor Sun, 22 Aug 2010 19:24:23 +0000 http://blog.codalism.com/?p=938#comment-14439 @Original blog post Man, I agree on your point that computer science is the "engineering arm of mathematics" in that it bends mathematics to fulfull certain tasks, but I wouldn't use the wording you did, I'd definitely not say computer science is "mathematical engineering". So, Computer Science is about bending mathematics to specific tasks, but so is Operations Research, which is considered an applied mathematics discipline rather than an engineering discipline. Computer Science is not much different from Operations Research at all, and I think it should qualify as Applied Mathematics too. Someone stated that Computer Science should be called Software Engineering, and that's just wrong. Software Engineering is concerned with working under the limitations of technology to create digital software and systems according to specifications, and Computer Science, on the other hand, is more concerned with the mathematics underlying the computation of results, not restrained to the existing computer technology. Software Engineering suffers from engineering-type constraints of feasibility, viability, etc. while Computer Science's constraints are mathematical possibility and human imagination. @Original blog post

Man, I agree on your point that computer science is the “engineering arm of mathematics” in that it bends mathematics to fulfull certain tasks, but I wouldn’t use the wording you did, I’d definitely not say computer science is “mathematical engineering”.

So, Computer Science is about bending mathematics to specific tasks, but so is Operations Research, which is considered an applied mathematics discipline rather than an engineering discipline. Computer Science is not much different from Operations Research at all, and I think it should qualify as Applied Mathematics too.

Someone stated that Computer Science should be called Software Engineering, and that’s just wrong. Software Engineering is concerned with working under the limitations of technology to create digital software and systems according to specifications, and Computer Science, on the other hand, is more concerned with the mathematics underlying the computation of results, not restrained to the existing computer technology. Software Engineering suffers from engineering-type constraints of feasibility, viability, etc. while Computer Science’s constraints are mathematical possibility and human imagination.

]]> Comment on How (and why) not to rank academics by required http://blog.codalism.com/?p=1220&cpage=1#comment-12746 required Sun, 01 Aug 2010 03:06:21 +0000 http://blog.codalism.com/?p=1220#comment-12746 Thanks for the explanation, it seems some people are "tool makers" - they produce TREC and other reports; some people are "trail blazers" - they push the state of the art with geniune research ideas, published in peer-reviewed venues; and yet there are others, just playing the game of citation. hopefully there are more metrics to reveal the true contribution of scholars. Thanks for the explanation, it seems some people are “tool makers” – they produce TREC and other reports; some people are “trail blazers” – they push the state of the art with geniune research ideas, published in peer-reviewed venues; and yet there are others, just playing the game of citation.

hopefully there are more metrics to reveal the true contribution of scholars.

]]> Comment on Ad-hoc retrieval: measurably going nowhere by SIGIR: Research vs. Reality « MobBlog http://blog.codalism.com/?p=1029&cpage=1#comment-12389 SIGIR: Research vs. Reality « MobBlog Mon, 26 Jul 2010 10:56:16 +0000 http://blog.codalism.com/?p=1029#comment-12389 [...] by x%”) rather than look at novel problems (see #1 above). However, while doing so, there is no evidence of long-term, cumulative progress in decades of publications. On the other hand, I continue to miss [...] [...] by x%”) rather than look at novel problems (see #1 above). However, while doing so, there is no evidence of long-term, cumulative progress in decades of publications. On the other hand, I continue to miss [...]

]]> Comment on SIGSIGIR by SIGIR: Research vs. Reality « MobBlog http://blog.codalism.com/?p=1148&cpage=1#comment-12388 SIGIR: Research vs. Reality « MobBlog Mon, 26 Jul 2010 10:55:18 +0000 http://blog.codalism.com/?p=1148#comment-12388 [...] chose to publish public rebuttals to the reviewers on their blog; others wrote about the unending cycle of complaining that the IR community has spiraled into. “Not Relevant” was born in the wake of all [...] [...] chose to publish public rebuttals to the reviewers on their blog; others wrote about the unending cycle of complaining that the IR community has spiraled into. “Not Relevant” was born in the wake of all [...]

]]> Comment on How (and why) not to rank academics by william http://blog.codalism.com/?p=1220&cpage=1#comment-12232 william Sat, 24 Jul 2010 00:14:44 +0000 http://blog.codalism.com/?p=1220#comment-12232 TREC participant reports are very rarely cited, with a small number of exceptions, such as the BM25 report from (I think) TREC 3. Overview papers are widely cited so that there is a reference for a data set that is being used. This is somewhat similar to the practice of citing a paper that describes a tool you're using in your research; for instance, the Terrier team gets a lot of cites for a SIGIR workshop paper describing the Terrier retrieval system by people who use that retrieval system. Such cites perform three chief purposes. First, they point the reader to a location where they can get more information on a dataset or tool. Second, they serve as an acknowledgment and sort of "citation-tip" to the people who developed the dataset or tool. And third, it's simply good form to have references in academic papers. What such cites do not do, though, in the great majority of cases, is refer to the research contribution of the paper being cited. There is merit in counting such citations; they reward groups that make resource contributions to the research community. But resource contributions are not the same as research contributions. It is therefore misleading to describe researchers as "top authors" in a research field because of citations to description documents for resources they've released. Where, then, to draw the line? Well, any line is going to be somewhat arbitrary; but the line traditionally draw in the research community is at peer-reviewed venues. And TREC is not. TREC participant reports are very rarely cited, with a small number of exceptions, such as the BM25 report from (I think) TREC 3. Overview papers are widely cited so that there is a reference for a data set that is being used. This is somewhat similar to the practice of citing a paper that describes a tool you’re using in your research; for instance, the Terrier team gets a lot of cites for a SIGIR workshop paper describing the Terrier retrieval system by people who use that retrieval system. Such cites perform three chief purposes. First, they point the reader to a location where they can get more information on a dataset or tool. Second, they serve as an acknowledgment and sort of “citation-tip” to the people who developed the dataset or tool. And third, it’s simply good form to have references in academic papers. What such cites do not do, though, in the great majority of cases, is refer to the research contribution of the paper being cited.

There is merit in counting such citations; they reward groups that make resource contributions to the research community. But resource contributions are not the same as research contributions. It is therefore misleading to describe researchers as “top authors” in a research field because of citations to description documents for resources they’ve released. Where, then, to draw the line? Well, any line is going to be somewhat arbitrary; but the line traditionally draw in the research community is at peer-reviewed venues. And TREC is not.

]]> Comment on How (and why) not to rank academics by required http://blog.codalism.com/?p=1220&cpage=1#comment-12204 required Fri, 23 Jul 2010 13:49:45 +0000 http://blog.codalism.com/?p=1220#comment-12204 layman's question - if a TREC paper has no value, why would people cite it? if very few people cite it, then why do we care? Hope to see some more explanations. TIA layman’s question – if a TREC paper has no value, why would people cite it? if very few people cite it, then why do we care?

Hope to see some more explanations.

TIA

]]> Comment on How (and why) not to rank academics by william http://blog.codalism.com/?p=1220&cpage=1#comment-12184 william Thu, 22 Jul 2010 23:07:10 +0000 http://blog.codalism.com/?p=1220#comment-12184 It took me a good half hour of poking around to figure out why MSAS was giving the rankings it was. How many people are going to make that effort? And I have the domain experience to understand why counting TREC citations is invalid. How many people have this expertise? By publishing these rankings, MS is implicitly giving their credit to them. When academics start writing "ranked 4th in Information Retrieval by MSAS" in their promotion applications, how many promotion committee members are going to take the time or have the expertise to investigate and (correctly) discount those claims? Microsoft Academic Search overall has merit, but the rankings don't. It took me a good half hour of poking around to figure out why MSAS was giving the rankings it was. How many people are going to make that effort? And I have the domain experience to understand why counting TREC citations is invalid. How many people have this expertise? By publishing these rankings, MS is implicitly giving their credit to them. When academics start writing “ranked 4th in Information Retrieval by MSAS” in their promotion applications, how many promotion committee members are going to take the time or have the expertise to investigate and (correctly) discount those claims?

Microsoft Academic Search overall has merit, but the rankings don’t.

]]> Comment on How (and why) not to rank academics by Irrelevant http://blog.codalism.com/?p=1220&cpage=1#comment-12178 Irrelevant Thu, 22 Jul 2010 20:55:55 +0000 http://blog.codalism.com/?p=1220#comment-12178 I imagine academics are quite capable of assessing the merits and limitations of this (or any) rank. Microsoft's effort has merit per se. I imagine academics are quite capable of assessing the merits and limitations of this (or any) rank. Microsoft’s effort has merit per se.

]]> Comment on Ad-hoc retrieval: measurably going nowhere by FXPAL Blog » Blog Archive » Maintaining relevance http://blog.codalism.com/?p=1029&cpage=1#comment-12142 FXPAL Blog » Blog Archive » Maintaining relevance Thu, 22 Jul 2010 07:04:40 +0000 http://blog.codalism.com/?p=1029#comment-12142 [...] do complex work, the quality of that work can be measured, and progress made. (Of course sometimes progress isn’t cumulative, but that’s a different [...] [...] do complex work, the quality of that work can be measured, and progress made. (Of course sometimes progress isn’t cumulative, but that’s a different [...]

]]>