\documentstyle{article}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input tcilatex
\QQQ{Language}{
American English
}

\begin{document}

\section{Comments from Bob}

Dear Amir,

I am in agreement with what Rick has to say although I am not so pessimistic
about the future or the paper.

If the paper is rewritten so it provides an even handed discussion of how
this algorithm works, it would be acceptable to me. But  it would be
acceptable to me {\bf only} in that form, unless we can show actual
improvement.

The problem is if we are as honest as I think we must be, the reviewers may
share Rick's opinion and choose not to publish the paper, in which case you
have wasted a lot of time (because of us, really because of me) and I would
feel very very badly.

\section{Specific Comments from Rick:}

Page 1: ``Wavelet analysis can be viewed as an alternative to classical {\it %
Windowed Fourier Analysis}.'' This is certainly true, but I do not see that
it has anything to do with the present paper. Windowed Fourier Analysis is
not (to the best of my knowledge) used for channel detection.

Page 1: ``This framework provides us with novel robust algorithms that will
apply state-of-the-art signal and stochastic processing techniques to some
of the cutting edge problems of molecular biology ..." This certainly seems
to overstate what is presented in this paper. By implication, it suggests
that what follows is likely to be better in some sense than previously used
techniques.

Page 2: ``Our results demonstrate the feasibility and utility of multiscale
analysis.'' In my opinion, the results do NOT demonstrate the utility of
multiscale analysis in the sense that nothing presented shows any obvious
advantage of this approach. The utility of this approach is assumed, but not
clearly demonstrated. 

I am not saying that multiscale analysis is not useful; I am saying its
utility has not been demonstrated in this manuscript.

Page 2: ``One advantage of wavelet analysis is that it is surprisingly
undemanding of computer resources and so can be implemented in real time.''
This has certainly not been demonstrated in the paper. Clearly from the
description of the algorithm, this approach must require much more computer
time than is required by simple threshold detection. If we wish to say
otherwise, we must support our opinion with results.

Page 3: ``We believe that the wavelet transform theory has a good chance to
succeed in many applications where the classical Fourier analysis has failed
or is not adequate." I do not believe that this has anything to do with this
paper.

Page 5: ``We examine the performance of our algorithm on models with white
noise and with filtered $f^2$ noise." This is the only time $f^2$ noise is
mentioned in this paper, none of the results appear to have come from
channel currents embedded in $f^2$ noise.

Page 5: ``It should be noted that the filtering rounds the corners of the
signal, spreading them on about 1.7 sample points." Delete.

Pages 5-7: ``Training for Level Estimates -- Preprocessing Step''. I wonder
what is gained from this section. There are certainly a variety of ways of
accomplishing the level estimation, and it is not demonstrated that this is
the best or quickest method. It would not work in the presence of any
significant drift and so might have to be supplemented in many real
application.

Page 8, Figure 1: As I have already pointed out, operationally, all that is
being done here is really a high pass filter (differentiation) followed by a
series of low pass filters, with the output of each low pass filter being
what is used at each level. This should be stated.

Page 8: ``We perform several cycles of Gaussian wavelet smoothing by
applying the low-pass part of the wavelet transform." This is a needlessly
confusing way of stating that the data was simply passed through simple
Gaussian low-pass FIR filters. In fact, the three filters used could have
been replaced by a single filter (3 Gaussian filters with -3 dB bandwidths
of 1, 2, and 4 used in series can be replaced with a single filter with a -3
dB bandwidth of 0.873).

Page 8: ``In the future we plan to eliminate the smoothing procedure in the
beginning in order to detect short events.'' This should either actually be
tried or not it should not be mentioned at all. Since without smoothing the
SNR will initially be much worse, eliminating smoothing actually may not
work.

Page 9: ``The well known Canny edge detector ... " This is not well known to
Physiologists.

Page 9: What is a Lipschitz exponent? This is not well known to
Physiologists.

Page 11: ``we detect the modulus maxima by finding the points where the
discrete wavelet transform is larger than its two closest neighbor values
and strictly larger than at least one of them''. This sounds to me like it
will detect {\sf every }noise peak (including very small ones), as well as
signal peaks. This would seem time consuming and not very efficient. Perhaps
I do not completely understand; operationally, what exactly is being done?

Page 11: I do not understand what criterion is used for ``chaining'' points
across several scales. $E.g.$, it is stated that ``locations 6000, 6001,
6000, 5999, 5990, 5977 are chained according to the algorithm already
described''. I understand that each location is from a different scale, but
the locations differ by up to 24 points (6001 vs. 5977). {\sf How is it
determined that these all represent the same feature?}

Page 12, Table 2: I still do not understand why the $a_i{}^{ref}$ values
increase as the level number increases. What sort of normalization is being
used? Obviously, since each new level can be obtained simply by low pass
filtering the previous level, the amplitude of peaks would normally decrease
with additional filtering.

Page 12, equation 11: why $is<r_i{}^{ref}\times 0.2$ selected, $i.e.$, why
0.2?

Page 14: Why only start considering the value of $A_{max}$ at level 4?

Page 15: ``Threshold verification" What is the point of this section? I
think it is unnecessary.

Page 15: ``This procedure $[i.e.$, threshold detection] can work
independently well in certain cases for single ionic channels. But it is
limited to single channel, while the proposed algorithm is more general in
the sense that it can be extended to two or more ionic channels." Threshold
detection can readily be extended to more than one channel.

Page 20, ``Experimental results": This section is very weak. Equations 14
and 15 are too obvious to need to be included. No significant results are
actually presented.

Page 21, Table 4: It seems that the only difference in the rows is the open
channel amplitude (2.0 going down to 1.10). The number of levels of
smoothing is always 3, and the {\sc rms} noise before smoothing is always
0.87; however, the {\sc rms} noise {\it after} smoothing is different in
each case. How is this possible? In the first row the {\sc rms} noise
actually {\it increases} (from 0.87 before smoothing to 0.90 after
smoothing); again how is this possible? Looking at the last two rows, the
value of A is 1.10 in each case and the {\sc rms} noise is 0.87 before
smoothing in each case, but the {\sc SNR} before smoothing is reported in
one case to be 1.26 and in the other case 1.14. Clearly in both cases it is
1.26. Perhaps A in the last column was intended to be 1.0. Even so, there
seems to be something wrong with this table. In addition, none of the
``results'' listed tabulate anything about the performance of anything but
the training algorithm, and the very questionable results of a smoothing
operation. In the results illustrated in Figures 3-6, smoothing increases
the {\sc SNR} from 1.45 to roughly 12; why does smoothing have so little
effect in Table 4?

Page 23: ``Better smoothing and detection should be possible, for example,
using a Harr (rectangular function) basis." Without some justification for
this statement, it should be eliminated.

Page 23: ``... oversample in general, hoping that the steep nonlinear
dependence of wavelet resolution on number of samples ...'' This must be
justified or eliminated. There is no obvious reason this should help in this
situation.

Page 23, paragraph starting with ``Another possibility is to eliminate the
smoothing procedure ...'' Again, justify this or eliminate the entire
paragraph. Mentioning ``more sophisticated denoising algorithms'' without
any hint of what they are adds nothing to the paper.

Page 23: ``In a later stage, we plan to study the decay of the noise across
the scales of a multi-resolution analysis.'' This can be done trivially by 
simple analytical techniques.

Figures 7-9: At no place in these figures is the utility of multiscale
resolution clearly demonstrated. Every accepted event would have been
accepted at level 2; there may be some improvement in terms of locating the
edge correctly, but this is not clear.{\bf \ This does not mean that the
multiscale aspect of the algorithm does not help, but simply that it has not
been demonstrated. }Some demonstration{\sf \ that multiscaling helps} would
seem necessary, since requiring that features `survive' through many levels
can, in fact, cause errors. It can cause brief events to be missed.

I continue to wonder whether the ``remarkable orthogonality properties'' of
wavelets have anything to do with what is actually done here in this paper.
I don't see why they would.

In order to make the paper represent what has been done (without depending
on things we have not done), some comments concerning the performance of the
algorithm relative to more traditional approaches would seem appropriate.
The paper should also use some plain English to describe operationally what
actually has been done and what is actually being presented. 

I think several things are important:

1. Much of the algorithm can be replaced with a high pass filter followed by
a series of low pass filters. In fact, this is what it is.

2. High pass filtering (differentiation) reduces signal to noise ratio for
channel data provided that the noise is white or has a PSD that increases
with increasing frequency. This is a penalty associated with this type of
edge detection. I can provide details of this later if you wish.

3. Something is required to demonstrate that the multiscale aspect of the
algorithm significantly improves results.

4. Something is also required to demonstrate that this approach has any
advantages at all over more traditional methods. Nothing presented (except
implication) suggests that it does:

Improved signal to noise handling ability is NOT demonstrated. In fact,
noise handling ability of this algorithm should be somewhat poorer than
thresholding.

The amount of computer time required for the algorithm is NOT less than
traditional thresholding. It seems clear that this algorithm will be slower
than thresholding.

The algorithm does NOT proceed without prior knowledge of channel current
amplitude. In this sense it is identical to thresholding.

The training algorithm is NOT better than a number of alternative approaches.

Because of the need for training the overall algorithm is NOT insensitive to
drift. This throws out one of the main advantages of differentiating the
data.

If some possible advantages of this approach can be defined, then it can be
considered worthwhile. If such advantages can not be defined and discussed,
then there is no real contribution here. An exploration of what wavelets
have to offer to channel detection is worthwhile. However, this paper in its
present form appears to indicate that the answer is ``Nothing''. 

If there are advantages then they should be stated (they are not in this
paper). If not, then the negative conclusion is the result of the
exploration, and, if anything at all is published, it should clearly state
this conclusion. The assertion that wavelets have great promise in this
application needs to be backed up; at present this is an article of faith
rather than an established fact.

\section{General Comments from Rick on Manuscript}

Overall, I am not very happy with the manuscript. The paper generally
successfully describes the algorithm used (although sometimes in a rather
confusing way). However, nothing is presented that suggests that there is
any advantage to using this algorithm. Of course, there is the usual amount
of hyperbole, such as ``This framework provides us with novel robust
algorithms that will apply state-of-the-art signal and stochastic processing
techniques to some cutting edge problems ... '', but nothing presented backs
up such assertions. Indeed, nothing whatsoever in the way of any
`benchmarks' concerning the performance of the algorithm is presented. Yet
it is obvious at a glance from Figures 5 or 6 that the results presented
could have been easily matched with {\bf simple} thresholding.

\section{Rick's Comments on Amir's letter of October 30:}

I certainly agree that it is not necessary for the method described to
presently be able to out-perform more traditional approaches in any
particular area of performance $(e.g.$, noise handling ability) in order to
be worthwhile---provided that {\it some} advantage of the new approach is
demonstrated. If no advantage is presented, then I am much less convinced
that there is any contribution from the method. Thus the description of some
sort of advantage of the approach seems important to the paper. It is
certainly not sufficient (in my opinion) to assert without evidence that
future improvements are somehow inevitable.

If the assertion ``We can even find examples that defeat the threshold
method" is true, then I would strongly urge that one or more such examples
be presented. So far, I have not seen such an example.

The actual algorithm used is not altogether new to channel detection.
Differentiation followed by low pass filtering has certainly been used
previously, and is hardly a novel idea. Mutiresolution analysis (to the best
of my knowledge) has {\em not} been used previously for channel detection;
however, its utility must actually be demonstrated (not just assumed).

It is certainly a mistake to assert that years of effort have been invested
in the threshold method of channel detection. It is certainly true that this
method has been used for years, but it is a very simple and straight-forward
method that no one has spent years developing. The actual time spent
developing the technique for channel current detection is probably very
limited. It is also definitely true that ``the threshold method seems to
have reached its upper bound''. But it did so a long time ago. It is a
simple method, whose performance can be predicted quite precisely. This has
been true for many years. However, it has not been demonstrated in the case
of channel current detection that ``multiresolution has a big future
ahead'', or that ``its potential is huge''. This may be true---or it may
not. What I think is needed is some demonstration of this future or
potential, not just faith that it exists.

To have achieved results that are comparable (or, in fact, somewhat worse)
than threshold analysis with another technique in a period of two months is
a reasonable accomplishment, but certainly not an extraordinary one. There
are a variety of possible approaches that can yield results that are
comparable to those presently achieved in most labs. What I strongly feel is
needed is an explanation of the possible advantages of any new technique.
Such a discussion should be concise and concrete and not rely on possible
future improvements unless the likelihood of their success can be clearly
demonstrated.

\end{document}