response-one.tex

% \setlength{\paperwidth}{210mm}
% \setlength{\paperheight}{297mm}% fixed.

\documentclass{article}

\usepackage[review]{myreviewpckg}

\usepackage{textcomp}
\usepackage{simplemargins}
\usepackage{ifpdf}

\ifpdf
	\usepackage[pdftex,bookmarks=false]{hyperref}
\else
	\usepackage{graphics}
	\usepackage{graphicx}
	\usepackage{epsfig}
	\usepackage[dvipdfm,bookmarks=false]{hyperref}
\fi

\hypersetup{
    pdfpagelayout=OneColumn,
	colorlinks = true,
	urlcolor   = blue,
	citecolor  = blue,
	linkcolor  = blue
}

\clubpenalty=10000  % Orphan - First of paragraph left behind
\widowpenalty=10000 % Widow  - Last of paragraph sent ahead

\setallmargins{1in}

\begin{document}

% *****************************************************************
% *********************** Associate Editor ************************
% *****************************************************************

\thispagestyle{empty}
\setcounter{page}{0}

~

\vspace{6em}
\noindent\today

\vspace{5em}
\noindent
Dr.~Thomas Brocher\\
Associate Editor, BSSA\\
BSSA Editorial Office\\
400 Evelyn Ave, Ste 201\\
Albany, CA 94706

\vspace{4em}
\noindent
Dear Dr. Brocher,

\vspace{1em}
\noindent
Thank you for coordinating the review of our manuscript (BSSA-D-18-00056) and for your own comments about it. Altogether, they have certainly helped improve our submission to the Bulletin. We are enclosing a response to both your suggestions and those of the reviewers. We were able to satisfy most, if not all of them. To help in this second review cycle, in the annotated manuscript version of the paper, we have highlighted the most relevant changes that resulted from your comments and those of the reviewers in green font; and below we provide a response to each one of your comments, followed by our responses to the reviewers. In all three cases we reproduce the original comments in italic black font, and our responses in regular blue font.\\

\noindent
We look forward to hearing back from you.

\vspace{2em}
\noindent
Sincerely yours,


\vspace{5em}
\noindent
Ricardo Taborda\\
Department of Civil Engineering, and\\
Center for Earthquake Research and Information\\
The University of Memphis

\newpage
\begin{center}
	\bf
	\large
	Authors' Response to the Associate Editor
\end{center}

\noindent
We thank the Associate Editor for his comments and suggestions. Below, we provide a response to each comment. The original comments are in italic black font, followed by our responses in regular blue font.
\vspace{2ex}
\newline

\introcomment{~}{%
I agree with the peer reviewers that this paper addresses a topical subject of great importance to BSSA readers. The science is in the process of transitioning from purely observational based analysis of ground motions to a mixture of observations and simulations of ground motions. The paper is a useful contribution to this discussion, is generally well written, and carefully describes the methodology used and its limitations. The peer reviewers have made some very helpful suggestions for improving the paper that I hope the authors will strongly consider.
}

\response{%
Thank you. We were able to address all the suggestions made by the reviewers. In the vast majority of the cases we adopted their suggestions and we only disagreed with a very small portion of them, in which case we presented our arguments in the response to the reviews enclosed here.
}

\introcomment{~}{%
In particular, I agree with their comment that the limitations of the study, i.e., the choice of 11 metrics and earthquake simulations for only one earthquake, needs to be included in the abstract and the conclusions.}

\response{%
We modified the abstract and the conclusions accordingly. In the case of the abstract we added minor modifications to emphasize the use of 11 metrics, and a whole new sentence explicitly referencing the use of three simulation datasets corresponding to the 2008 Chino Hills earthquake. In the case of the conclusions, although we already had a paragraph describing the limitations of the study, we added a small sentence reemphasizing it.
}

\introcomment{~}{%
In addition, I also agree that all the terms and methodology should be well defined for the BSSA readership, so that the paper can be understood by them without resort to frequent online searches.
}

\response{%
Thank you for this suggestion. We revised the \textbf{Introduction}, \textbf{Study Dataset}, and \textbf{Data Analysis} sections to carefully introduce and explain the different terms and their equivalencies, when necessary. We think the introduced modifications should solve this issue and provide a long-lasting reference for the BSSA readership.
}

\introcomment{~}{%
I found Figure 6 very helpful in giving me a very simple if heuristic understanding why C4, C5, C6, and C8 are the most diagnostic of the goodness of fit. It clearly shows that only these metrics are change significantly with the goodness of fit (P, F, G, E). Plotting these values as a function of P, F, G, and E together on the same figure might allow the reader to see this relationship more clearly and might help provide a heuristic understanding of how C4, C5, C6, and C8 are diagnostic whereas C11 is constant with P, F, G, and E, and thus not diagnostic. In fact, in my opinion, Figure 6 is a simple and easy way of determining which metrics are diagnostic that might even eliminate the need for the decision trees. In any case, more discussion of this figure might help the authors make the case for the main conclusions.
}

\response{%
Thank you. This is very true. Figure 6 certainly conveys a clear message about how the metrics vary for each validation classes. The decision tree analysis, nonetheless, is still needed in order to convert this qualitative insight into a set of quantitative conditionals to decide the final class based on the GOF scores. In attention to this comment, we modified Fig.~6 to highlight the boundaries of the categories defined by Anderson (2004) and expanded the text in the Results section to provide a better explanation of the meaning of the results shown in this figure.
}


% ******************************************************************
% *************************** REVIEWER 1 ***************************
% ******************************************************************

\newpage
\begin{center}
	\bf
	\large
	Authors' Response to Reviewer 1
\end{center}

\noindent
We thank the reviewer for his/her comments and suggestions, which helped us improve the manuscript. To aid the review process, in the annotated manuscript version of the paper, we have highlighted the most relevant changes that resulted from these comments in green font. Here, we provide a response to each comment. The original comments are in italic black font, followed by our responses in regular blue font.
\vspace{2ex}
\newline

\introcomment{~}{%
This is an interesting article for the readers of the BSSA. The authors propose a new approach based on machine learning techniques to determine the importance of validation metrics for ground motion simulation. The field of ground motion simulation validation (GMSV) is relatively new and of interest to both seismologists who develop simulation models and engineers who would like to use these simulations in engineering applications. One of the most challenging questions to address is ``which validation metric(s) is best to use?'' as there are so many different metrics in this field and so many different opinions. It has been difficult to address this question because the answer varies depending on the application of the simulations (e.g., GMPE development, PSHA, structural analysis response of interest, etc). This paper is interesting because it provides a statistical approach to reduce the number of validation metrics and select a few representatives from a bigger set without compromising the final validation results. The authors have done a great job providing background on the topic. However, their application is very narrow, they only consider the 11 Anderson2004+ parameters (as they have noted, there are other parameters) for one simulation method for one earthquake, and as a result their conclusions are only valid for this particular case. They have mentioned this limitation in the conclusions, but the abstract and introduction do not explain the limitations and the application-dependency of the proposed approach. Both Abstract and Introduction need modification to specify the application for which the results and conclusions are true (i.e., the application here is to narrow down the 11 Anderson2004+ parameters to select fewer parameters that give similar results as the original 11 for the simulations they chose in this paper). Currently, the Abstract and Introduction give the impression that the conclusions are true for any engineering application or simulation method. Furthermore, there should be a short discussion in the paper on how the users can apply the same method for different applications, for example for structural responses such as drift ratio or responses that are sensitive to high-frequency content of simulations (theoretically, this method should be applicable to any response of interest if they replace the GOF scores used in this study? If true this should be mentioned as it will be of interest to others).
}

\response{%
We thank the reviewer for the positive feedback in this general review. We agree that there were some important points that we did not discuss fully in the manuscript and we have corrected them. In attention to this general review, we modified the abstract, the introduction, and the conclusions to (i) transparently point out the limitations of our study, and (ii) offer an additional perspective on the potential refinement of it or use with additional datasets or in reference to additional validation metrics.
}

\comment{1}{%
The terminologies used in the paper are not clear in many cases. For example, what are ``features'' ``attributes'' ``dimensions'' ``trees, nodes and leafs'' ``class''? It was very difficult to read the section on Data Analysis Method and Introduction since these terminologies were not clearly defined or described in terms of the variables used in this paper. I was confused what ``data'' and ``results'' were referring to: GOF score data? GOF metric data? GOF category? Something other than GOF? I suggest that the authors streamline the paper and the terminologies to be consistent and clear in the final version.
}

\response{%
We revised the \textbf{Introduction}, \textbf{Study Dataset}, and \textbf{Data Analysis} sections to carefully introduce and explain the different terms and their equivalencies.
}

\comment{2}{%
The terms ``common practice'' or ``common standards'' have been mentioned several times. Because GMSV is such a new field (and I don't think there is consensus, I for one don't agree with some statements but I agree that others may), I think these terms should be removed. Just say this is your opinion, or it is based on the references that you have listed.
}

\response{%
Thanks for the comment. We limited the use of these terms to a minimum and/or replaced them by sentences providing additional context.
}

\comment{3}{%
In abstract, ``semi-supervised'' and ``supervised'' learning techniques are mentioned, but in the paper these terms are not explained, what part of the learning techniques is semi-supervised and what does that mean?
}

\response{%
Thank you for bringing this up. We now provide a better approach to understanding these terms in the introduction and also included small changes to highlight the differences in the Data Analysis Method section.
}

\comment{4}{%
In introduction, the difference between ``verification'' and ``validation'' should be discussed.
}

\response{%
A brief description of the differences between verification and validation was added to the Introduction.
}

\comment{5}{%
In abstract and introduction, what does a ``direct application'' mean? it's not clear. Similarly, in Validation Metrics section: ``direct quantitative comparisons'' and ``indirect statistical analysis''.
}

\response{%
The term ``direct application'' was present in the Introduction, but it has been removed due to improvements in the text based on other suggestion by this and a Reviewer \#2. The terms ``direct quantitative comparisons'' and ``indirect statistical analysis'' were present in the Introduction and the Validation Metrics sections. In our view, the first of these terms makes reference to validation through direct, i.e., signal to signal, quantitative comparisons of metrics; while the second of these terms makes reference to validation by means of statistical analysis showcasing the overall results of comparisons, and thus the use of ``indirect''. We understand, however, that this may not be a view shared by any other reader, so we have modified the text to be more explicit about this distinction.
}

\comment{6}{%
Page 4, line 53: add ``relatively'' before ``well understood''.
}

\response{%
We appreciate the suggestion made by the reviewer. In attention to it, we modified the sentence with a choice of words that we have used in two previous papers published in BSSA, which convey the same idea we wanted to transmit but in a slightly different way. That being said, our view is that there should be nothing relative about the fact that these metrics are well understood by seismologists and engineers, and the strength of Anderson's because of this has been pointed out before by others.
}

\comment{7}{%
The two questions posed in the introduction lines 59--61: I don't think the first one is really the question, all parameters should be considered if anyone has thought of them as important (i.e., has proposed them). The second question is the main question and the one being addressed in this paper.
}

\response{%
We thank the reviewer for expressing his/her opinion about this point. First, we changed slightly the sentences as a whole to emphasize these are indirect questions as opposed to direct questions. That said, we do think they both are valid and relevant questions. The argument made by the reviewer that any or all parameters should be included if someone has thought of them as important is precisely one of the points we argue against, and the results of this research support that argument. The fact that someone thinks of a parameter as relevant does not guarantee its relevance, it is only the reflect of his/her expert (though still subjective) opinion. There may be parameters that are more influential, and these can be as influential as to the point of proving others irrelevant. This is the advantage of the procedure we follow here, which allows one to evaluate objectively, based on how data relates to the results, how relevant or irrelevant is each one of the metrics involved in reaching the same conclusions. This, in turn, allows one to prioritize the metrics, which is the point of the second indirect question posed in this part of the introduction.
}

\comment{8}{%
You are treating these metrics as ``independent variables'', but as you have also noted many of them are not. For example, Arias intensity and energy are correlated. Does this compromise your conclusions? If not explain why in the Results section.
}
%
\response{%
This is a very valid point. However, an important aspect to keep in mind is that the clustering process and the subsequent construction of the decision is not the same as a correlation analysis, and it is therefore not compromised by having highly correlated metrics. On the contrary, the process naturally selects the metrics with the best chance of predicting the outcome. In the case of two metrics being highly correlated, the process would still analyze which of the two is a better predictor. In some cases it may decide to keep both, and in some cases it may decide that one is enough. In attention to this and other related comments we decided to include an additional table (Table 7) showing how the level of participation of the different metrics would change if one removes some of the initially most relevant metrics. This should help the readers better understand the results of our work.
}

\comment{9}{%
Page 7, line 133: delete the last sentence
}

\response{%
We followed this suggestion and deleted the sentence.
}

\comment{10}{%
Page 14, line 310: you say ``...the latter can essentially be discarded...'' I don't understand this statement, wouldn't this mean that both C1 and C7 are important to consider since they are independent of each other (one gives information that the other one doesn't)?
}

\response{%
In the specific case of the 2-feature clustering in the C1-C7 subspace, what the results show is that only C7 (in the vertical axis) is playing a role in establishing boundaries between the different classes. C1 does not help to do that. C1 may be important in other context, but here it does not add any value to the decision making process. That is why we call it irrelevant feature, because one can define the clusters based on only defining boundaries on C7. It is normal to have a tendency to interpret this as a correlation problem, in which the interpretation of independence (i.e., lack of correlation between C1 and C7) would indicate that both are relevant. But that is not the goal of the clustering process. The goal is to identify which metrics play a role in delimiting the classes when correlated to other metrics. In other words, the colors and the way boundaries end up forming in these figures, is as important as the shape of the clouds of data-points.
}

\comment{11}{%
Page 14, line 323: It's not clear how you ``replicate data'', provide more detail.
}

\response{%
We modified this paragraph to provide more details about the oversampling process.
}

\comment{12}{%
I don't think ``confusion matrix'' given in tables is explained in the paper!
}

\response{%
The explanation about confusion matrices in the original manuscript was in the section titled \textbf{Decision Trees}, page 13, between lines 280 and 282. The explanation included an illustrative example for the simplest case of two categories. We assumed it would be clear that this concept is extendable to larger matrices such as those shown in Tables 2 and 3. We added a sentence in the explanation to make it explicit that they can be extended to deal with more attributes.
}

\comment{13}{%
Page 17, lines 391--394: this sentence should be re-worded, what is an ``accurate conclusion''? You should be very specific that your conclusion is that instead of using all 11 validation metrics you can select a few in order to get the same categorization as all 11 metrics would have given you.}

\response{%
You are right. It could be misleading. We reworded the second half of this paragraph to make things more transparent and to keep them in the context of what we used in our study.
}

\comment{14}{%
Page 18, line 405: this is implying that fig 10 is similar to fig2! I personally cannot see this in the figures! They look very different. It might be better to regenerate fig 11a but with different color scales that only vary by four choices, then the comparison would be much easier.
}

\response{%
In hindsight, we may have allowed ourselves to be carried away by our own optimism. We have therefore modified our comments in this section to a more self-critical approach about the results. The question is really not whether they look the same or not, but whether they lead to similar---if not equivalent---conclusions. It is our interpretation that they do in the sense that looking at the results obtained with either approach one may walk away with the same conclusion about the simulation as a whole and about how well (or not well) did the simulations turned out with respect to specific areas in the simulation domain. We modified the text to convey this message, now hopefully more clearly than before.
}

\comment{15}{%
Figure 8: I don't know what you mean by y-label ``decision attributes''. Do you mean ``the number of GOF metrics''? why there is no data point for 1 or 11?
}

\response{%
That is right. It should be the number of decision attributes. We modified the figure accordingly. The reason why there is no data for 1 is because the analysis did not yield any tree that would classify the data using a single metric. On the other end, the reason there are no trees with the 11 metrics is because the analysis of the data resulted in the metric C11 not having any participation in the decision process, as shown in Table 6. We also tried other trees for which we removed metrics C5 and C8. In such cases the participations of the metrics changes and so do the topology of the trees. Some of the trees resulting from that analysis did include the metric C11, but with a very small participation, as shown in the new Table 7.
}

\comment{16}{%
The conclusion basically says that response spectrum is the most important validation metric. This is not surprising to any engineer or modeler. The big question is what other parameters describe the differences between a simulation and a recorded motion if their response spectra matched? Or in other words, what are the other important validation parameters aside from response spectrum (many have argued that's duration, but no statistical proof). Can you use your method to answer this question? Can you condition your machine learning process on Sa being the same (for example use a subset of data with similar Sa or artificially make them equal), then decide what's the next important parameter? Would the result be the same as in your conclusions? Or would it be influenced by removal of Sa as a parameter (this is possible since there are correlations between Sa and other metrics).
}

\response{%
These are very good points. Yes, the general consensus about the importance of the response spectra (C8) may not be surprising, but it is still very useful to confirm such importance through a data-informed process. Most important, as pointed out by the reviewer, is to answer the question about what other parameters are relevant. We think our work contributes precisely to that. We mention in the conclusion that the next metrics in line are the energy (C4), the peak acceleration (C5) and the peak velocity (C6). In order to further emphasize this point, as described in the response to comment \#8, we also included a new table (Table 7) showing the participation of the different metrics in the absence of the response spectra (C8) and the peak acceleration (C5). The results confirm that in such cases the next most relevant metrics are still the energy (C4) and the peak ground velocity (C6), as mentioned before. This additional analysis also reafirms the initial conclusion about the very small (and in some cases null) participation of the duration (C11) in most of the decision trees built in the data-processing analysis carried out here.
}

\noindent
\textbf{Typos:}

\vspace{2em}

\comment{1}{%
Page 2, line 8: change ``focused'' to ``focus''.
}

\response{%
Done. Thanks.
}

\comment{2}{%
Page 2, line 20: change ``classifiers'' to ``classifier''
}

\response{%
In fact, we were referring to ``these decision trees'', so we removed the ``a'' before ``data-informed qualifiers'' instead. The plural form is now consistent. Thank you.
}

\comment{3}{%
Page 13, line 285: change ``measure'' to ``measured''.
}

\response{%
We changed it. Thanks.
}

\comment{4}{%
Page 16, line 354: the last C5 should be C6?
}

\response{%
Yes, we fixed it.
}

% ******************************************************************
% *************************** REVIEWER 2 ***************************
% ******************************************************************

\newpage

\begin{center}
	\bf
	\large
	Authors' Response to Reviewer 2
\end{center}

\noindent
We thank the reviewer for his/her comments and suggestions, which helped us improve the manuscript. To help the review process, in the annotated manuscript version of the paper, we have highlighted the most relevant changes that resulted from these comments in green font and the removed text in red font. In the following, we provide a response to each comment. The original comments are in italic black font, followed by our responses in regular blue font.
\vspace{2ex}
\newline

\introcomment{~}{%
Earthquake ground motion simulations are usually validated against observational data or empirically-derived solutions. However, exactly which parameters to compare (e.g., peak ground velocity, duration, spectral content) in order to assess the goodness-of-fit, is not always clear. This paper uses machine learning techniques and eleven possible metrics for comparing (recorded and simulated) ground motions to determine the parameters that are most predictive of goodness-of-fit. This is an interesting paper, and will help future researchers prioritize which metrics to focus on when validating ground motions simulations. The paper is generally well written.
}

\response{%
We thank the reviewer for the generally positive view of our paper.
}

\introcomment{~}{%
As a strong ground motion seismologist (i.e., I assume I'm part of the audience you hope to reach), there is a lot of machine learning-related jargon that makes certain sections difficult to follow.
}

\response{%
Thank you for this suggestion. We revised the \textbf{Introduction}, \textbf{Study Dataset}, and \textbf{Data Analysis} sections to carefully introduce and explain the different terms and their equivalencies, when necessary.
}

\introcomment{~}{%
It would also be useful to have some discussion regarding which parameters were found to be repetitive, in case future authors wish to look at more than the four recommended parameters.
}

\response{%
We addressed this by considering additional trees in the cases of removing metrics C5 and C8 (see Table 7). This was also done in response to the comments \#8 and \#16 from Reviewer 1. As we explain in the response to those comments, the procedure used here is different from a simpler correlation analysis in which a metric can be directly related by other if ``repetitive''. Here, the algorithm searches for the potential of each metric to serve as a predictor. In some cases, one may need two well correlated metrics to aid in that decision process, while not in others. By including the results shown in the new Table 7, instead of trying to understand these correlations, we can see what metrics become relevant in the absence of others. The results indicate that in addition to the energy and the response spectra, the next in line are the peak response in acceleration, velocity, and the Arias intensity. They also confirm the lack of relevance of the duration (C11), the time integrals (C1) and (C2), and the cross correlation (C10).
}

\comment{1}{%
Line 8: ``... could \textbf{focus} ...''
}

\response{%
Done. Thanks.
}

\comment{2}{%
Line 11: ``...the objective \textbf{of}...''
}

\response{%
Added. Thanks.
}

\comment{3}{%
Lines 25--41: The frequent use of phrases such as ``Among these methods,''; ``Among the latter,''; ``In this category,''; and ``Within this group,'' make the text difficult to follow.
}

\response{%
We modified the second half of the first paragraph in the Introduction to be more direct in the description of the alternative verification and validation metrics.
}

\comment{4}{%
Lines 42--43: I would suggest moving the description of the Anderson (2004) method (i.e., Line 48+) to directly follow this sentence.}

\response{%
Thanks for the suggestion. We moved it and it does read better this way.
}

\comment{5}{%
Lines 59--61, 391--394: It should be stated somewhere that this is really intended for \textbf{engineering purposes}.
}

\response{%
While it is true that the initial intent of Anderson (2004) was to develop credibility of synthetic seismograms for engineering applications, we do not share the need to make that intent explicit here. On the one hand, these metrics have already been used by several authors who did not intend to validate seismograms for any engineering application, but to simply show---by means of them---that their synthetics were realistic in a general sense. On the other hand, we do not want to convey the idea that the steps that we followed are exclusive to narrowing validation criteria for engineering applications. Others may see our work and these steps as applicable to other ends. That being said, we hope the reviewer will be pleased to notice that we modified the Introduction to point out that engineering applications are the one of the motivating factors for the increased attention in verification and validation. Please see the first sentence in the introduction (see line 29).}

\comment{6}{%
Line 61: This sentence should end with a ``?''.
}

\response{%
In our view, these sentences are written as indirect questions and do not require the use of a question mark. To make this clearer, we modified the beginning of the sentences slightly by removing the colon and adding the word ``regarding''.
}

\comment{7}{%
Lines 98--99: It is not clear how the equation for $S$ is applied to values that are a function of period (e.g., the response or Fourier spectrum). Do you follow the guidelines provided by Anderson? If so, what frequency range/bands are you examining?
}

\response{%
Yes, we follow the guidelines of Anderson (2004), but the question of the reviewer gives way to two separate points of those guidelines: first, that in general the equation for $S$ is applied to all values of frequency (or period) for which is computed and then averaged to produce the GOF scores for C8 and C9; and second, that one can do this process for the broadband signals as well as for different frequency bands, and then average the results. The latter of these two points is applicable to not only C8 and C9 but to all metrics. In this study we only use broadband results to avoid the adding other parameters to the validation process such as the width of the frequency bands or the characteristics of the filters. We added a clarification about how the method guidelines were used to address this comment.
}

\comment{8}{%
Lines 128/132/137/etc.: ``... the three component\textbf{s}...''
}

\response{%
Fixed. Thanks.
}

\comment{9}{%
Line 236: Where is the discussion on the consequences of overfitting? The only other place I see overfitting mentioned is line 328.
}

\response{%
We added it at this point. Thank you.
}

\comment{10}{%
Line 236: Remove ``A matter we discuss later.''
}

\response{%
We removed it, and in its place added an explanation about overfitting in response to the previous comment.
}

\comment{11}{%
Lines 241--243: It is not helpful to list different algorithm names for making decision trees, without any information about them or how they differ.
}

\response{%
Although it could certainly be good to have a comprehensive discussion about different methods, that would take much space, and it is not within the scope of this study. Instead, we provide adequate references to suggest further reading for the interested audience. That being said, in attention to the Reviewer's comment, we modified slightly the descritpion to give some additional context in what regards to the C5.0 algorithm and added a sentence explaining that detailing the alternative algorithms is out of the scope of this study.
}

\comment{12}{%
Lines 257--258: I don't understand this sentence. What is a continuous attribute?
}

\response{%
Following Equation (3) we define $A[a]$ as all the possible values of attribute $A$. The use of $[a]$ instead of $(a)$ implies that $A$ is considered to be discrete. We therefore meant to say that if $A$ was continuous, as in $A(a)$, then the reader should refer to Quinlan (1996). We improved the text to make this clearer.
}

\comment{13}{%
Lines 259--264: This paragraph contains a lot of jargon.
}

\response{%
Indeed. We expanded the paragraph to include an explanation about the options used with the C5.0 algorithm.
}

\comment{14}{%
Line 300: Isn't this also true for the C5-C7 combination?
}

\response{%
That is right. We modified the sentence to reflect that.
}

\comment{15}{%
Line 328: What is ``a strong pruning process''?
}

\response{%
We modified it to ``heavy pruning process'' and added an explanation earlier in the Decision Trees section.
}

\comment{16}{%
Line 433: ``We present the result\textbf{s} of...''
}

\response{%
Added. Thanks.
}

\comment{17}{%
Line 435: ``...goal of prioritiz\textbf{ing} and reduc\textbf{ing}...''
}

\response{%
Indeed. We modified them accordingly. Thanks.
}

\comment{18}{%
Line 443: Remove ``Similarly'' or ``similar'' (redundant).
}

\response{%
We eliminated ``Similarly''. Thanks.
}

\comment{19}{%
Line 450: ``...remain valid.''
}

\response{%
The extra ``s'' was deleted. Thanks.
}

\comment{20}{%
Line 455: ``...as the decisive \textbf{parameters}...'' ? (This short concluding paragraph would be much stronger without the use of unnecessary phrases, i.e., ``This latter point'', ``One the one hand'', ``on the other hand.''
}

\response{%
Indeed. We modified the last paragraph of the Conclusions by removing those sentences.
}

\end{document}